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at  lower  levels  of  English  language  proficiency.  Furthermore,  verbal  response 
justifications  for  incorrect  answers  fE-xperiments  1  and  3)  indicated  that  the 
majority  of  errors  were  due  either  to  inaccurate  processing  of  the  text  or  to  an 
inability  to  choose  the  connector  that  fit  an  inappropriately  inferred  relation. 

The  results  are  discussed  in  terms  of  the  processing  demands  of  the  various  types 
of  connectors  and  the  influence  of  everyday  usage  of  causal  and  additive  connectors 
on  how  individuals  use  and  understand  such  terms.  Implications  for  improving  the 
design  of  content-domain  texts  are  provided  and  instructional  strategies  for  use 
with  ESL  students  are  suggested. 


Knowledge  of  Connectors  as  Cohesion  Devices  in  Text:  A 
Comparative  Study  of  Native  English  and  ESL  Speakers 
A  primary  method  of  acquiring  new  information  is  through  reading.  This  process 
involves  understanding  the  relations  among  the  clauses  and  sentences  that  make  up  a  passage, 
and  results  in  a  representation  of  the  information  that  is  organized  and  logically  coherent. 

There  are  several  types  of  rhetorical  devices  that  may  be  used  in  text  to  make  explicit  the 
logical  structure  of  the  information.  Some  devices  signal  the  global  organization  of  the  passage 
(e.g.,  titles,  headings  and  subheadings)-,  others  indicate  the  function  of  a  paragraph  (e.g.,  In 
summary)-,  and  others  function  at  a  more  local  level  conveying  the  relationship  between 
successive  clauses  or  sentences  in  a  text  (e.g.,  predicate  connectors  such  as  however,  in 
addition,  because).  (For  discussions  of  signalling  see  Lorch,  (in  press)  and  Meyer  (1975).) 

Passages  vary  in  the  degree  to  which  the  inter-  and  intrasentential  relations  contained 
therein  are  made  explicit.  In  the  absence  of  explicit  signals  to  logical  relationships  we  rely  on  a 
"conversational  postulate"  for  text:  we  infer  cohesion  based  on  the  default  assumption  that 
successive  sentences  are  related,  making  use  of  other  cohesion  devices  such  as  referential  and 
lexical  overlap  (Halliday  &  Hasan,  1976;  Kintsch  &  van  Dijk,  1978;  MacLean  &  D’Anglejan, 
1986).  Research  on  the  role  of  prior  knowledge  in  text  comprehension  suggests  that  the 
outcome  of  this  inference  process  depends  on  the  amount  of  knowledge  the  reader  has  in  the 
domain  of  the  discourse  (  Goldman  &  Dur£n,  1988;  Voss,  1984).  When  the  reader  has  little 
knowledge  in  a  domain  and  is  reading  to  learn  about  the  domain,  the  presence  of  signals  that  cue 
logical  relationships  among  sentences  would  seem  to  be  particularly  important. 

Connectors,  or  conjunctives  (Halliday  &  Hasan,  1976),  are  a  type  of  cohesion  device 
that  make  explicit  the  logical  relations  among  sentences.  Common  connectors  include  and,  but. 
however,  and  because.  Such  connectors  are  of  limited  utility,  however,  unless  the  reader 
understands  how  connectors  function  in  text  and  the  logical  relationship  each  specifies.  This 
aspect  of  language  proficiency  is  particularly  important  to  students  who  acquire  English  as  a 
second  language  (ESL)  and  are  attempting  to  learn  from  English  language  texts.  Success  for 


these  students  in  the  university  setting  is  in  part  determined  by  the  ability  to  learn  from  text. 
One  focus  of  the  present  research  was  the  levels  of  proficiency  with  connectors  that  might  be 
characteristic  of  ESL  populations.  Although  we  were  able  to  find  only  one  study  that  examined 
ESL  students'  use  of  connectors  to  establish  cohesion  in  text  (MacLean  &  d'Anglejan,  1986), 
anecdotal  reports  from  teachers  of  ESL  students  frequently  indicate  that  learning  the 
appropriate  use  of  connectors  is  an  extremely  difficult  aspect  of  the  English  language.  To  a 
lesser  extent,  this  can  also  be  said  of  native  English  speakers.  For  example,  in  a  study  of  good 
and  poor  readers.  Bridge  and  Winograd  (1982)  found  that  both  groups  found  it  more  difficult  to 
justify  cohesive  ties  established  with  connectors  compared  to  those  established  with  referential 
and  lexical  cohesion  devices;  however,  good  readers  were  better  able  to  justify  their  responses 
than  were  poor  readers. 

Studies  examining  the  use  of  connectors  by  readers  of  different  skill  levels  have  yielded 
mixed  results  regarding  the  relationship  between  connector  understanding  and  language 
proficiency.  Some  studies  have  found  that  the  presence  of  connectors  in  text  is  no  more  helpful 
to  good  readers  than  to  poor  readers  (e.  g.,  Meyer,  Brandt,  &  Bluth,  1980)  but  others  report 
differential  effects.  For  example,  Geva  and  Ryan  (1985)  found  that  positive  effects  of  including 
connectors  occurred  for  both  groups  when  the  conjunctions  were  included  and  highlighted  in  the 
text  but  only  for  the  skilled  readers  when  the  conjunctions  were  present  in  the  text  but  not 
highlighted.  Furthermore,  Geva  and  Ryan  (1 985)  found  that  the  omission  of  connectors 
negatively  affected  less-skilled  readers  but  had  no  effect  on  skilled  readers. 

Other  researchers  have  examined  whether  difficulty  level  of  the  text  impacts  the  role  of 
connectors  and  have  found  no  relationship  (Spyridakis  &  Standal,  1987).  MacLean  &  d'Anglejan 
(1986)  examined  the  role  of  text  difficulty  on  advanced  ESL  learners'  ability  to  establish 
intersentential  coherence.  They  presented  students  with  a  rational  cloze  task  in  which  three 
types  of  cohesive  items  had  been  deleted,  including  conjunctive  items.  (The  rational  cloze  task 
contrasts  with  the  traditional  cloze  task  where  the  deletion  scheme  is  based  on  number  of 


intervening  words  (e.g.,  delete  every  fifth  word).  MacLean  &  d'Anglejan  found  that  text 
difficulty  did  not  affect  advanced  ESL  learners'  ability  to  complete  the  rational  cloze  task. 
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The  equivocal  nature  of  the  results  of  prior  research  regarding  language  proficiency  and 
connectors  is  in  part  due  to  the  sampling  of  connectors.  It  seems  reasonable  that  some 
interclausal  and  intersentential  relations  may  be  easier  to  infer  than  others  and  the  benefits  of 
including  explicit  connectors  may  be  differential.  In  addition,  grammar  textbooks  for  ESL 
students  typically  devote  large  sections  to  sentence  and  clause  connectors,  providing  elaborate 
taxonomies  of  connectors.  For  example,  Celce-Murcia  and  Freeman  (1983)  have  provided  a 
functional  classification  scheme  for  logical  connectors  that  differentiates  among  four  major 
types  of  connectors:  additive,  causal,  adversative,  and  sequential.  The  first  three  typically 
relate  to  interclausal  or  intersentential  relations.  Additive  relations  are  those  that  signal  some 
form  of  elaboration  of  previous  content  (e.  g.,  In  addition,  That  is).  Cause-effect  or  antecedent- 
consequent  relations  are  signalled  by  causal  connectors  (e.g.,  As  a  result,  Due  to).  Adversative 
connectors  signal  contrastive  elaborations  (e.g.,  However,  On  the  contrary).  Sequential 
connectors  are  used  to  signal  a  more  diverse  set  of  logical  relations  and  refer  to  the  larger 
discourse  context  more  often  than  do  the  other  connectors.  Sequential  connectors  are  used  to 
enumerate  lists  of  items  (e.g.,  First,  Second,  Finally),  to  mark  a  sentence  that  previews  the 
remainder  of  the  text  (e.g.,  In  short )  or  to  indicate  temporal  sequence  of  events  (e.g., 
Subsequently,  Later).  In  previous  studies,  investigators  have  not  systematically  controlled  or 
manipulated  the  relation  that  the  reader  must  infer.  For  example,  Spyridakis  and  Standal 
(1987)  sampled  a  heterogeneous  set  of  logical  relations.  In  other  cases,  effects  of  different 
kinds  of  connectors  remain  unknown  because  the  sample  of  specific  connectors  was  not  indicated 
and/or  data  were  collapsed  across  all  connectors  (e.g.,  Bridge  &  Winograd,  1982;  Geva  &  Ryan, 
1985;  MacLean  &  d'Anglejan,  1986). 

In  the  present  series  of  studies  we  examined  college  students'  understanding  of  the  use  of 
connectors  by  having  students  complete  a  rational  cloze  task  in  which  the  four  previously 
mentioned  connector  types  served  as  multiple  choice  alternatives.  We  wished  to  determine 
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whether  the  four  types  of  logical  connectors  were  differentially  difficult  for  readers  to 
understand.  Our  expectations  were  that  sequential  connectors  would  be  more  difficult  than  the 
other  three  types  of  connectors  for  all  types  of  readers  because  sequentials  generally  require 
reference  to  the  more  global  discourse  context  whereas  the  other  three  do  not.  The  prediction  of 
increased  difficulty  is  based  on  the  finding  that  comprehension  difficulty  increases  when 
establishing  coherence  requires  reference  to  more  "distant"  prior  input  (e.g.  van  Dijk  & 
Kintsch,  1983).  Among  the  other  three  connector  types,  all  indicate  some  form  of  continued 
elaboration  of  previous  information.  However,  they  differ  with  regard  to  what  they  signal  is  the 
connection  of  new  information  to  previous  information,  with  causal  and  adversative  connectors 
signalling  more  specific  relations  than  additives.  The  latter  indicate  only  that  the  next  piece  of 
information  will  be  adding  "in  some  way"  to  the  previous  information.  In  contrast,  as  noted 
previously,  causal  and  adversative  connectors  respectively  signal  causal  or  logical  contingency 
or  contrast  between  old  and  new  information  in  the  clauses  joined  by  the  connectors. 

Predictions  about  the  relative  difficulty  among  these  three  connectors  were  not  made. 

We  were  also  interested  in  the  strategies  that  readers  used  to  arrive  at  their  choices  and 
whether  these  were  similar  for  native  English  and  ESL  readers,  especially  where  differential 
levels  of  English  language  proficiency  were  present.  In  prior  research,  Bridge  and  Winograd 
(1982)  found  that  good  readers  tended  to  rely  on  both  within-  and  across-sentence  information 
when  explaining  connector  cloze  completions  whereas  poor  readers  tended  to  use  one  qi  the 
other.  However,  MacLean  &  d'Anglejan  found  that  both  ESL  and  native  English  speakers  relied 
on  across-sentence  information  when  performing  rational  cloze  tasks.  MacLean  and  d'Anglejan 
(1986)  also  reported  that  ESL  readers  used  similar  across-sentence  strategies  in  their  first 
and  second  languages.  We  pursued  the  strategy  question  somewhat  differently  than  MacLean  and 
d’Anglejan  (1986)  by  focusing  on  whether  response  justifications  reflected  the  differences 
among  the  connectors  that  are  cited  by  text  linguistics,  discourse  analysts,  and  ESL  teachers. 

We  collected  retrospective  verbal  reports  from  students  explaining  their  choices.  Our 
predictions  were  that  the  native  English  speakers  would  be  more  sensitive  to  the  differences 
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among  the  connectors  and  this  would  be  reflected  in  their  response  justifications,  as  well  as  in 
the  frequency  of  correct  choices.  On  the  other  hand,  because  ESL  students  tend  to  be  taught 
prescriptive  rules  for  connector  usage,  their  justifications  for  correct  choices  were  also  likely 
to  be  appropriate. 

Three  experiments  are  described  in  this  report.  The  studies  compared  the  performance 
of  native  English  speakers  and  ESL  students  in  several  populations  that  vary  in  level  of  language 
proficiency  in  English.  In  the  first  experiment,  we  compared  cloze  task  and  verbal  justification 
performance  of  monolingual,  native  English  speakers  and  ESL  students  in  a  population  of 
university  students.  Based  on  the  findings  from  Experiment  1  of  differential  difficulty  among 
the  connector  types,  the  purpose  of  Experiment  2  was  to  determine  if  monolingual,  native 
English  speakers'  confidence  in  their  answers  also  varied  with  connector  type.  Finally,  in 
Experiment  3,  a  group  of  community  college  ESL  students  completed  the  rational  cloze  task, 
provided  confidence  ratings  and  gave  verbal  justifications  for  their  responses. 

EXPERIMENT  1:  Effects  of  Connector  Types  on  the  Cloze  Completions  and 
Response  Justifications  of  Native  English  and  ESL  Speakers. 

The  purpose  of  Experiment  1  was  to  compare  the  performance  of  native  English  and  ESL 
university-level  college  students  on  a  rational  cloze  task  that  required  them  to  distinguish 
among  four  logical  connectors  in  choosing  the  most  appropriate  intersentential  connector.  The 
rational  cloze  procedure  was  applied  to  passages  characteristic  of  introductory-level  textbooks 
used  in  university  settings.  Our  general  prediction  was  that  ESL  students  would  make  fewer 
appropriate  choices  than  native  English  speakers.  We  expected  the  differential  difficulty  of  the 
four  types  of  connectors  to  be  similar  in  each  group  of  students.  Finally,  based  on  previous 
research  we  expected  that  ESL  and  native  English  speakers  would  be  similarly  aware  of 
prescriptive  rules  on  the  usage  of  the  various  connector  types  but  that  the  ESL  students  would 
be  less  accurate  identifying  where  in  text  the  various  rules  applied. 
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Method 

Subjects.  • 

Participants  were  16  monolingual,  native  English  speakers  and  20  ESL  speakers.  All 
students  were  recruited  from  undergraduate  classes  at  a  university  in  Southern  California.  The 
native  English  speakers  were  enrolled  in  Introductory  Psychology  and  participated  as  part  of  the  • 

course  requirements.  The  ESL  students  were  enrolled  in  English-language  classes  specifically 
designated  for  nonnative  speakers  of  English.  Successful  completion  of  these  classes  is  a 
prerequisite  to  satisfying  the  university's  General  Education  English  requirement.  ESL  students  • 

who  participated  in  this  study  were  volunteers  recruited  from  classes  at  three  levels,  but  the 
majority  (15  students)  came  from  the  class  (English  1)  one  level  below  the  university- 
required  English  class.1  ESL  students  were  paid  $5  per  hour  for  their  participation.  • 

Several  demographic  characteristics  were  assessed  using  a  questionnaire  filled  out  at  the 
beginning  of  the  experimental  session.  A  summary  of  information  most  pertinent  to  the  present 
study  is  provided  in  Table  1.  The  native  English  speakers  were  largely  college  freshmen  and  # 

sophomores  and  all  spoke  English  as  the  native  language.  The  mean  verbal  Scholastic  Aptitude 
Test  (SAT)  Score  of  526.8  indicates  that  these  students  were  representative  of  the  entering 
freshmen  classes  at  this  university  over  the  past  several  years.  The  ESL  students  were  largely  • 

freshmen  and  sophomores.  The  native  language  of  70%  of  the  ESL  students  was  an  Asian 
language  (Chinese,  Vietnamese,  Japanese  or  Korean).  The  verbal  SAT  scores  for  the  ESL 
speakers  are  typical  for  this  population.  Performance  on  the  Test  of  English  as  a  Foreign  # 

Language  (TOEFL)  is  the  language  proficiency  indicator  typically  used  in  making  admissions 
decisions.  The  mean  score  of  564.7  is  representative  of  students  from  non-English  speaking 
countries  who  are  admitted  to  the  university.  • 


Insert  Table  1  about  here 
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As  an  extremely  rough  indicator  of  background  knowledge  about  the  passage  topics,  the 
questionnaire  asked  students  to  indicate  how  many  high  school  and  how  many  college-level 
courses  they  had  taken  in  domains  related  to  the  content  of  the  passages  (e.g.,  biology, 
oceanography,  history,  sociology,  anthropology).  Most  of  the  students  reported  having  taken  2 
or  3  courses  in  these  areas.  Thus,  there  was  little  variability  among  the  students  and  all  could 
be  regarded  as  relatively  "low  knowledge"  in  the  domains  discussed  by  the  passages.  The  majors 
reported  by  the  native  English  and  ESL  students  indicated  an  interesting  difference  between 
groups.  The  majority  of  the  ESL  students  were  majoring  in  the  natural  sciences,  including 
engineering.  In  contrast,  half  of  the  native  English  speakers  were  undecided  about  their  choice 
of  major,  with  only  18%  indicating  natural  sciences. 

Design  and  Materials 

The  design  was  a  mixed  factorial  with  one  between-subjects  factor  (native  English  or 
ESL  language  group)  and  one  within-subjects  factor  (connector  type).  There  were  four  levels 
of  connector  type:  Additive,  Causal,  Adversative,  and  Sequential.  Each  connector  type  was  the 
correct  response  twice  in  each  of  4  passages  for  a  total  of  8  observations  per  subject  on  each 
connector.  In  addition  to  examining  the  number  correct,  we  asked  subjects  to  explain  their 
choices  for  two  of  the  passages.  Passages  were  presented  in  four  different  orders,  according  to  a 
catin-square  design.  Within  each  language  group,  each  presentation  order  was  used  an  equal 
number  of  times. 

Passages.  Four  passages  from  college-level,  introductory  science  and  social  science 
textbooks  were  modified  to  accommodate  2  instances  of  each  of  the  connector  types.  The  mean 
number  of  words  per  passage  was  488,  with  the  shortest  being  427  words  in  length  and  the 
longest  537;  the  mean  number  of  sentences  was  32.75,  with  the  shortest  passage  having  29 
sentences  and  the  longest  35.  For  2  of  the  passages  the  Flesh  grade  level  equivalent  was  9-10 
and  for  2  it  was  11-12.  The  passages  were  about  biological  characteristics  of  life, 
anthropological  approaches  to  the  study  of  culture,  oceanographic  explorations  in  the  nineteenth 
century,  and  the  societal  functions  of  marriage.  For  purposes  of  illustrating  the  task  to 


8 

subjects,  an  additional,  shorter  passage  (about  the  movie  industry)  was  developed.  It  contained 
one  instance  of  each  connector  type  and  was  at  grade  level  7-8  on  the  Flesch  scale. 

Subjects'  task  was  to  read  each  passage  and  to  choose  an  appropriate  word  to  fill  in  each 
of  the  cloze  slots  (represented  as  blanks)  that  occurred  throughout  the  passage.  Each  cloze  slot 
occurred  at  the  beginning  of  a  sentence  and  four  alternatives  were  provided.  Each  half  of  each 
passage  contained  four  slots  and  each  slot  required  a  different  connector  type  as  the  correct 
answer.  A  minimum  of  one  complete  sentence  separated  successive  slots.  The  alternatives  were 
instances  of  the  four  connector  types;  :he  correct  response  for  each  cloze  slot  was  determined  by 
the  experimenters.  That  the  experimenter  designated  response  was  the  best  choice  for  the  slot 
was  verified  by  a  panel  of  three  independent  judges,  one  of  whom  is  an  ESL  specialist.  The 
same  connector  type  could  not  be  the  correct  choice  for  two  successive  slots,  and  the  sequence  of 
connectors  as  correct  choices  was  different  in  each  half  of  the  passage,  as  veil  as  across 
passages.  No  one  connector  type  appeared  consistently  in  the  same  cloze  slot  across  passages. 

The  order  of  the  alternatives  was  systematically  varied  so  the  correct  choice  occurred  in  each 
position  (1,  2,  3,  or  4)  an  equal  number  of  times.  The  order  of  the  detractors  was  also  varied 
so  instances  of  the  same  connector  type  did  not  always  appear  in  the  same  position. 

Our  classification  of  connector  types  and  the  specific  instances  used  as  response  options 
were  taken  from  the  functional  classification  scheme  proposed  by  Celce-Murcia  and  Freeman 
(1983)  and  based  on  the  earlier  work  of  Halliday  and  Hasan  (1976).  For  the  additives  we  used 
simple  additive  instances  (e.g.,  in  addition)  and  exemplification  instances  (e..g.,  for  example ); 
for  causals,  we  used  instances  that  signal  cause/reason  (e.g.,  as  a  result)  and  effect/result 
(e.g.,  thus).  For  adversatives  we  used  instances  that  signal  that  two  ideas  are  in  contrast  or 
conflict  (e.g.,  however)  and  instances  that  signal  a  reservation  or  restriction  in  the 
applicability  of  the  preceding  information  (e.g.,  despite).  Sequential  connectors  featured  the 
largest  variety  of  subtypes  and  we  used  instances  that  signalled  enumeration  of  points 
chronologically  (e.g.,  second)  or  temporally  (e.g.,  next)  as  well  as  those  that  indicated 
summation  of  information  (e.g.,  in  short).  Table  2  shows  the  specific  instances  that  served  as 
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the  correct  response  for  each  connector  type  across  the  four  passages.  The  number  of  times  a 
specific  instance  served  as  distractor  as  well  as  as  a  correct  response  was  equalized  as  much  as 
possible.  Where  inequalities  in  the  frequency  of  occurrence  were  necessary,  we  used  those 
instances  that  occur  more  frequently  in  standard  texts. 

Insert  Table  2  about  here 


The  passages  (including  the  practice  passage)  were  arranged  in  a  single  "passage" 
booklet,  and  the  alternatives  for  the  slots  for  each  passage  were  arranged  in  a  separate 
"response"  booklet.  The  order  of  the  four  passages  was  counterbalanced  such  that,  across  all 
subjects,  each  passage  appeared  in  each  position  in  the  booklet  an  equal  number  of  times.  To 
avoid  differential  cueing  of  the  various  connectors,  all  punctuation  usually  associated  with  the 
use  of  the  connectors  (e.g.,  a  comma)  was  omitted. 

Procedure.  Subjects  were  run  individually  in  sessions  lasting  1 .5  to  2  hours.  Each 
subject  completed  a  background  questionnaire  at  the  start  of  the  session.  After  filling  out  the 
questionnaire,  each  subject  was  instructed  on  how  to  coordinate  the  passage  and  response 
booklets  and  worked  through  the  practice  passage  during  which  time  they  could  ask  questions 
about  the  task.  F :r  each  blank,  subjects  circled  the  best  word  to  complete  the  sentence  from 
among  the  four  alternative  words  that  were  provided  in  the  response  booklet.  Subjects  were 
told  that  all  punctuation  associated  with  the  alternatives  had  been  intentionally  omitted,  but  that 
they  could  add  any  punctuation  they  felt  was  necessary  to  their  response. 

Following  completion  of  the  four  passages,  the  experimenter  re-presented  the  first  two 
passages  (excluding  the  practice  passage)  the  subject  had  read.  Subjects  were  told  that  we  were 
interested  in  how  they  had  decided  on  their  answer  and,  in  why  they  thought  their  answer  was 
the  best  choice  for  the  particular  blank.  For  each  blank,  the  students  explained  their  reasons 
for  selecting  the;r  alternative  choices.  This  part  of  the  session  was  audiotaped  and  later 


transcribed. 


Scoring 


Each  subject  received  a  score  from  0  to  8,  indicating  the  number  of  correct  responses  to 
the  8  cloze  slots  for  each  connector  type.  We  determined  the  frequency  with  which  each  of  the 
three  distractor  alternatives  was  selected  incorrectly.  For  the  incorrect  responses  we  also 
examined  the  contingency  relation  between  the  correct  connector  and  the  type  of  distractor 
chosen.  In  the  verbal  justifications,  students  tended  to  describe  the  type  of  relationship  that 
held  between  the  items  of  information  that  were  being  connected.  To  score  these  responses,  a  set 
of  11  coding  categories  was  developed.  Descriptions  of  the  categories  were  initially  developed 
by  the  second  author  who  used  them  in  scoring  20%  of  the  protocols.  A  second  rater  used  the 
descriptions  to  score  the  same  20%.  Interrater  reliability  was  70%.  The  two  raters  then 
discussed  and  revised  the  category  descriptions  (provided  in  Appendix  A)  and  independently 
coded  a  new  20%  sample.  Interrater  reliability  for  the  second  sample  was  above  90%.  The 
second  rater  then  scored  all  of  the  protocols.  The  categories  reflect  the  distinctions  among  the 
connectors  described  in  the  introduction  and  each  coding  category  was  appropriate  for  only  one 
of  the  connector  types. 

Results 

Correct  Responses 

The  number  correct  for  each  subject  for  each  connector  type  was  subjected  to  a 
multivariate,  split-plot  ANOVA  in  which  language  group  was  the  between-subjects  factor  and 
connector  type  the  within.2  The  means  for  each  connector  type  and  language  group  are  provided 
in  Table  3.  As  expected,  the  main  effect  for  language  group  was  significant,  £  (1,  33)  =  15.28, 
B_<  .001,  Mserror  =  5.02:  The  native  English  speakers  were  correct  more  often  (M.  =  6.34) 
than  the  ESL  students  (M  =  4.88).  In  addition,  there  was  a  main  effect  for  connector  type, 

£  (3,  32)  =  3.27  fi_=  .03.  Five  post  hoc  contrasts,  using  the  Bonferroni  procedure  (alpha 
level  »  .01),  were  conducted.  Additives  and  causals  were  correctly  answered  more  often  than 
were  adversatives  and  sequentials,  £  (1,  34)  =  8.46,  a  =  .006,  Mserror  =  6.69.  The  other 


four  contrasts  were  not  significant  nor  was  the  interaction  between  language  group  and 
connector  type,  E  (3,  32)  =  1.58. 


Insert  Table  3  about  here 


Distractor.  Choices 

The  correct  response  data  indicate  that  the  ESL  speakers  made  more  errors  than  the 
native  English  speakers  -  a  mean  of  12.2  per  student  (out  of  32  possible)  compared  to  6.62. 
Regarding  these  errors,  the  following  issues  were  of  interest:  (a)  which  connectors  made  the 
best  distractors,  i.e.,  were  most  frequently  chosen,  and  (b)  whether  the  distractors  chosen  by 
the  two  groups  were  similar.  To  investigate  these  issues,  we  computed  the  proportion  of  each 
subject's  total  errors  that  reflected  each  distractor  option.  The  proportion  data  were  subjected 
to  an  arcsine  transformation  and  then  submitted  to  a  multivariate,  split-plot  ANOVA  with 
language  group  as  the  between-subjects  factor  and  connector  type  as  the  within.  There  was  no 
main  effect  for  language  group,  £  (1 ,  34)  =  3.04,  ft  «  .09.  However,  there  was  a  main  effect  of 
connector  type,  £  (3,  31)  =  10.06,  ft  <  .001:  Additives  and  causals  were  chosen  significantly 
more  often  than  adversatives  and  sequentials,  £  (1,  33)  =  9.48,  ft  =  .004. 

There  was  also  a  significant  language  group  by  connector  type  interaction,  E  (3,  31)  = 
4.69,  ft  »  .001.  Table  4  gives  the  mean  proportions  for  each  distractor  and  language  group. 
Examination  of  the  means  suggested  that  the  interaction  was  due  to  differences  in  the  magnitude 
of  the  differences  among  distractors.  Accordingly,  three  difference  scores  were  computed  for 
each  subject  in  each  group  and  the  differences  between  the  distractors  were  tested  for 
significance.  The  Dunn-Bonferroni  procedure  was  used  to  control  the  Type  I  error  rate  (alpha 
*  .017).  Causals  were  chosen  significantly  more  than  adversatives  by  each  group  but  the 
difference  was  larger  in  the  native  English  speakers,  1  (32)  =  2.83.  The  ESL  speakers  chose 
the  adversatives  and  sequentials  equally  often  but  the  native  English  speakers  chose  the 
sequentials  more  often  than  the  adversatives,  I  (32)  -  2.46.  Although  the  means  suggest  that 


the  native  English  speakers  chose  causals  more  frequently  than  additives  while  the  ESL  speakers 
chose  additives  more  than  causals,  the  difference  between  the  groups  failed  to  reach  conventional 
levels  of  significance,  i  (32)  =  1.88. 

Insert  Tables  4  &  5  here 

An  additional  analysis  pursued  potential  dependency  relations  between  the  errors  and 
the  correct  connectors.  The  presence  of  a  strong  dependency,  reflected  by  a  dominant  distractor 
for  a  particular  connector  type,  would  suggest  some  type  of  systematic  misunderstanding  of  the 
meaning  and  usage  of  that  connector.  The  proportions  shown  in  Table  5  reflect  the  number  of 
times  each  of  the  distractor  options  was  chosen  relative  to  the  total  number  of  errors  on  each 
connector.  These  dependency  matrices  indicate  a  relatively  strong  relationship  between  additive 
cloze  slots  and  causal  distractors  for  both  the  native  English  and  the  ESL  speakers  (probability 
of  choosing  a  causal  if  an  error  is  made  on  an  additive  cloze  slot  *  .76  and  .60,  respectively). 

For  each  of  the  other  connector  types,  incorrect  responses  tended  to  be  split  between  two 
distractor  options.  For  example,  both  additive  and  sequential  distractors  were  chosen  for  the 
causal  cloze  slots.  On  the  adversative  slots,  errors  were  evenly  split  between  additive  and 
causal  distractors.  The  native  English  and  ESL  students  differed  somewhat  in  their  choices  on 
the  sequential  cloze  slots:  When  the  native  English  speakers  made  errors  on  the  sequentials  they 
chose  the  additive  and  causal  distractors  equally  often.  When  the  ESL  speakers  made  the  wrong 
choice  on  the  sequential  slots,  they  had  a  greater  tendency  to  select  the  additive  connector 
(50%). 

Both  native  English  and  ESL  speakers  were  most  frequently  correct  when  additive  or 
causal  connectors  were  required  by  the  text;  they  also  chose  these  two  types  of  connectors  most 
often  when  they  responded  incorrectly,  and  there  was  some  evidence  of  a  dependency 
relationship  between  the  additive  cloze  slots  and  the  causal  distractor.  The  higher  correct  and 
distractor  choice  rates  for  additives  and  causals  suggests  the  possibility  that  subjects  were 


generally  biased  toward  choosing  additive  and  causal  terms.  We  examined  this  by  determining 
the  rates  of  choosing  each  of  the  four  connector  types  as  responses,  regardless  of  whether  the 
response  was  correct  or  incorrect.  Then  we  determined  the  percentage  of  choices  that  were 
indeed  correct.  These  selection  rate  data  are  presented  in  Table  6. 

Insert  Table  6  about  here 


According  to  the  design  of  the  passages,  each  connector  should  have  been  selected  25%  of 
the  time.  The  data  indicate  that  the  native  English  and  the  ESL  speakers  selected  the  causals  and 
the  additives  more  frequently  and  the  adversatives  and  sequentials  less  frequently  than  a 
"completely  correct"  response  profile.  However,  the  probability  of  being  correct,  given  the 
choice  of  a  particular  connector,  was  higher  for  the  adversatives  and  the  sequentials  than  for  the 
additives  and  causals.  It  appears  that  both  native  English  and  ESL  students  may  have  a  more 
stringent  set  of  constraints  governing  the  appropriateness  of  adversative  and  sequential  terms 
than  those  that  govern  additive  and  causals.  It  is  also  possible  that  adversative  instances  are 
less  familiar  (frequent)  than  the  causal  and  additive  instances. 

JustilicaliflnsjQf.ChQices 

Justifications  for  Choices  that  were  Correct.  Table  7  indicates  the  distribution  of 
correct-choice  justifications  over  the  coding  categories.^  The  proportions  were  computed 
separately  for  each  connector  type,  using  the  total  number  of  justifications  for  the  specific 
connector  as  the  base.  There  are  several  interesting  aspects  of  these  data.  First,  the 
distribution  of  justifications  for  the  ESL  students  is  highly  similar  to  that  of  the  native  English 
speakers:  for  each  connector  type,  the  proportional  distributions  over  the  coding  categories  for 
the  connector  reflect  the  same  pattern.  When  there  was  a  dominant  response,  as  in  the  case  of 
the  causals,  the  adversatives,  and  the  sequentials,  the  proportions  for  the  ESL  and  the  native 
English  speakers  were  almost  identical.  Second,  for  the  adversatives  and  the  sequentials  over 
95%  of  the  justifications  were  accounted  for  by  connector-appropriate  categories.  The 


proportion  of  connector-appropriate  justifications  was  somewhat  lower  for  the  additive  and 
causal  connectors,  although  more  than  75%  were  in  connector-appropriate  categories. 


Insert  Tables  7  &  8  about  here 

Justifications  for  Choices  that  were  Incorrect.  The  distributions  of  the  justifications  for 
incorrect  choices  are  shown  in  Table  8.  The  confusions  among  connectors  that  were  reflected  in 
the  contingency  analysis  of  the  errors  (Table  5)  were  generally  reflected  in  the  justification 
data.4  When  students  were  justifying  incorrect  responses  that  had  been  made  to  additive  slots, 
the  majority  of  the  justifications  were  in  categories  appropriate  to  causal  connectors.  This  is 
consistent  with  the  data  showing  that  causal  distractors  were  the  most  frequently  chosen  for 
additive  slots.  The  most  frequent  justifications  for  incorrect  choices  for  causal  slots  were  in 
categories  appropriate  for  additive  connectors,  with  the  remaining  responses  distributed  over 
the  causal-appropriate,  adversative-appropriate  and  miscellaneous  categories.  The  additive 
and  causal  distributions  for  native  English  and  ESL  speakers  were  highly  similar. 

There  were  some  differences  between  the  language  groups  in  the  distributions  of 
justifications  for  incorrect  responses  made  to  adversative  and  sequential  slots.  For  adversative 
slots,  the  native  English  speakers  used  categories  appropriate  to  the  causal  56%  of  the  time  and 
categories  appropriate  to  the  additive  13%  of  the  time.  In  contrast,  the  ESL  students  used 
causal-  and  additive-appropriate  categories  equally  often  (34%  each).  The  native  English 
speakers'  justifications  of  incorrect  choices  for  sequential  slots  tended  to  be  in  causal- 
appropriate  categories  most  frequently  and  additive-appropriate  or  sequential-appropriate 
categories  equally  often  (24%  each).  On  the  other  hand,  for  incorrect  sequentials  the  ESL 
students  used  additive-appropriate  categories  most  frequently,  causal-appropriate  less  often, 
and  rarely  used  sequential-appropriate  justifications. 

The  distractor  dependency  analysis  and  the  justification  data  considered  separately 
suggest  that  errors  may  be  due  to  a  "fuzzy"  understanding  of  certain  connectors.  To  more 


precisely  pursue  the  source(s)  of  difficulty  on  items  that  students  answered  incorrectly  we 
examined  the  relation  between  the  justification  and  the  connector  selected.  Three  relations  are 
informative  with  respect  to  source(s)  of  difficulty  and  these  are  illustrated  in  Figure  1 .5  The 
first  relation  -  incorrect  alternative  but  a  justification  appropriate  to  the  alternative  chosen  - 
indicates  that  the  error  is  due  to  difficulty  processing  the  information  in  the  text:  the  student 
has  selected  the  connector  that  matches  the  relation  extracted  from  the  text  but  it  is  not  the 
logical  relation  actually  called  for  by  the  text.  For  example,  students'  understanding  of  the  text 
may  have  made  them  think  that  a  causal  connector  was  needed  in  an  additive  slot;  students  chose 
the  causal  and  explained  their  choice  using  a  causal-appropriate  justification:  Thus,  they 
supplied  the  right  connector  for  the  wrong  relation.  This  was  the  dominant  pattern  for  both  the 
native  English  speakers  (63%  of  56  opportunities)  and  the  ESL  speakers  (65%  of  122 
opportunities). 

Insert  Figure  1  about  here 

The  second  type  of  relation  between  the  connector  selected  and  the  justification  - 
incorrect  alternative  but  with  a  justification  appropriate  to  the  cloze  slot  -  implies  difficulty 
with  the  meaning  of  the  connector  terms:  the  student  has  extracted  the  logical  relation  called  for 
by  the  text  but  does  not  choose  the  connector  that  conveys  that  relation.  For  example,  the 
student  explained  the  (incorrect)  choice  of  a  causal  saying  that  what  was  needed  in  the  text  was 
an  additive  connector.  This  pattern  did  not  account  for  many  of  the  errors:  16%  for  the  native 
English  speakers  and  6%  for  the  ESL.  The  third  relation  reflects  problems  in  processing  the 
text  and  in  matching  connector  terms  with  the  inferred  relation.  In  this  case,  the  student 
selected  an  incorrect  connector  and  provided  a  justification  that  was  appropriate  to  neither  the 
selected  connector  nor  the  cloze  slot  in  the  text.  This  pattern  accounted  for  a  moderate  amount  of 
the  native  English  speakers'  responses  (21%)  and  for  a  somewhat  higher  percentage  of  the  ESL 
students'  responses  (30%). 


Thus,  difficulty  extracting  the  appropriate  logical  relation  is  implicated  as  an  important 
source  of  student  errors:  inaccurate  inferences  about  the  appropriate  logical  relation  between 
information  contained  in  successive  sentences  in  the  text  accounted  for  the  vast  majority 
(95%)  of  the  ESL  students'  errors  and  84%  of  the  native  English  students'  errors.  The  data 
also  indicate  that  a  simple  "lack  of  knowledge"  of  the  functions  and  meanings  of  various 
connector-type  instances  was  not  a  primary  reason  for  incorrect  responses. 

Discussion 

The  results  indicate  that  native  English  speakers  correctly  completed  more  of  the  cloze 
slots  than  did  the  ESL  students.  For  both  groups,  the  pattern  of  difficulty  among  the  connector 
types  was  similar:  cloze  slots  requiring  additive  and  causal  connectors  were  more  likely  to  be 
filled  in  correct ly  than  were  cloze  slots  requiring  adversative  or  sequential  slots.  This  pattern 
of  differential  difficulty  partially  confirmed  our  expectations.  Although  we  had  expected  the 
sequentials  to  be  the  most  difficult,  we  had  not  expected  the  adversatives  to  be  as  difficult  as  the 
sequentials.  Justifications  for  correct  responses  were  similar  for  the  two  language  groups  and 
reflected  the  distinctions  among  connectors  that  we  had  postulated. 

When  incorrect  responses  were  made,  differences,  as  well  as  similarities,  emerged 
between  the  language  groups  and  among  the  connectors.  In  general,  when  native  English 
speakers  were  incorrect  they  completed  the  cloze  slots  with  causal  connectors  whereas  the 
general  tendency  for  the  ESL  students  was  to  choose  additive  connectors.  When  these  choices 
were  examined  contingent  on  the  correct  response  for  the  cloze  slots,  we  found  that  when 
additives  were  the  correct  choice,  both  language  groups  had  primarily  chosen  causals.  Both 
groups  showed  similar  choices  when  they  incorrectly  responded  to  causal  and  adversative  cloze 
slots.  However,  the  two  groups  differed  on  their  incorrect  choices  for  sequential  cloze  slots, 
with  the  ESL  students  primarily  choosing  additives  while  the  native  English  students  selected 
both  causals  and  additives. 

In  both  the  ESL  and  native  English  speakers,  there  were  tendencies  toward  overuse  of 
the  additive  and  causal  terms.  And  although  students  were  correct  most  frequently  on  additive 


and  causal  cloze  slots,  when  they  did  choose  adversative  and  sequential  instances,  there  was  a 
high  probability  that  these  choices  were  correct.  Taken  together,  these  findings  suggested  the 
possibility  that  students'  perceived  confidence  about  the  appropriate  logical  relationship  and 
choice  might  be  less  when  they  chose  additives  and  causals  as  compared  to  when  they  chose 
adversatives  and  sequentials.  We  pursued  this  issue  in  Experiment  2  with  native  English 
speakers  by  administering  a  confidence  rating  scale  along  with  the  forced-choice  cloze  task. 

EXPERIMENT  2:  Effects  of  Connector  Types  on  Cloze  Completions  and  Confidence 

Ratings  of  Native  English  Speakers 

Experiment  2  tested  the  hypothesis  that  students  would  be  more  confident  of  adversative 
and  sequential  cloze  slot  completions  than  of  additive  and  causal  completions.  This  study  also 
served  as  a  replication  of  the  basic  findings  of  Experiment  1  with  a  new  sample  of  native  English 
speakers.  The  methods  used  were  similar  to  those  of  Experiment  1 ,  except  that  confidence 
ratings  were  made  and  no  justification  data  were  collected. 

Method 

Subjects 

Thirty-two  native  English  speaking  undergraduates  enrolled  in  an  Introductory 
Psychology  class  at  a  university  in  Southern  California  participated  in  this  study  for  class 
credit.  The  demographic  data  reported  on  the  background  questionnaire  are  summarized  in  Table 
9.  Freshmen  and  sophomores  comprised  this  sample  of  native  English  speakers,  mean  age  was 
equivalent  to  the  native  English  group  from  Experiment  1 ,  but  the  verbal  SAT  score  was  about 
50  points  higher.  Subjects  had  taken  about  the  same  number  of  related  courses  as  the  students 
in  Experiment  1 ,  with  only  one  student  indicating  no  courses  in  any  of  the  passage-related 
areas.  A  somewhat  higher  percentage  of  these  students  were  majoring  in  the  Social  Sciences 
(43%)  and  only  28%  were  undecided.  The  percentage  of  students  majoring  in  natural  sciences 
and  engineering  was  lower  than  that  reported  by  the  ESL  sample  from  Experiment  1  but  was 
comparable  to  the  native  English  sample. 


Insert  Table  9  about  here 


Design.  Materials  and  Procedure 

The  materials  used  in  this  study  were  identical  to  those  used  in  Experiment  1  with  the 
addition  of  a  confidence  scale  that  was  printed  on  every  page  of  the  response  booklet.  This  scale 
was  printed  in  number  line  form  and  ranged  from  1  (very  low  confidence)  to  7  (very  high 
confidence).  Beneath  each  number  was  printed  a  word  reflecting  the  degree  of  confidence 
represented  by  that  number.  The  procedure  was  identical  to  that  of  Experiment  1  with  three 
exceptions:  1)  all  subjects  were  run  together  in  a  single  group  session;  2)  no  interviews  were 
conducted  with  these  subjects;  and  3)  after  deciding  which  alternative  was  correct  for  each  slot, 
subjects  were  instructed  to  rate  the  confidence  they  had  in  their  choice  by  circling  a  number 
from  1  to  7  on  the  scale  printed  below  the  alternatives. 

The  design  was  a  single  factor  within-subjects  design  in  which  connector  type  was  a 
four-level  factor. 

Results 

Correct  Responses. 

As  in  Experiment  1 ,  the  number  of  correct  responses  for  each  type  of  connector  was 
computed  for  each  subject.  The  results  of  a  one-way,  multivariate,  repeated  measures  Anova 
indicated  a  significant  effect  for  connector  type,  E  (3,  29)  -  8.86,  ft  <  .001.  Four  post  hoc 
contrasts  of  the  means  shown  in  Table  10  were  computed,  using  the  Bonferroni  procedure 
(alpha  =  .0125).  As  in  Experiment  1,  additives  and  causals  were  correctly  answered  more 
frequently  than  were  adversatives  and  sequentials,  £(1.31)  =  13.4,  a  <  .001,  Mserror  =  6.8. 
Causals  were  correct  more  frequently  than  additives,  E  (1.  31)  -  8.02,  a  =  .008,  Mserror  * 
2.43.  The  difference  between  adversatives  and  sequentials  was  not  significant,  E  (1,  31)  = 

1.37.  Also  shown  in  Table  10  are  students'  confidence  ratings  for  those  items  that  were  correct. 
A  multivariate  one-way  ANOVA  on  the  confidence  ratings  indicated  a  significant  effect  of 


connector  type,  £  (3,  29)  =  8.8,  p  <  -001.  Post  hoc  comparisons  of  the  additive  and  causal 
ratings  with  the  adversative  and  sequential  ratings  confirmed  our  prediction:  Confidence 
ratings  were  higher  for  correctly  chosen  adversatives  and  sequentials  than  for  additives  and 
causals,  £  (1,  31)  =  23.75,  p.  <  .001,  Mserror  =  -771. 


Insert  Tables  10  &  11  about  here 

Distractor  Choices. 

As  in  Experiment  1 ,  we  computed  the  distribution  of  each  subject's  total  errors  over  the 
distractor  alternatives.  These  proportions  were  transformed  using  the  arcsine  transformation 
and  submitted  to  a  one-way,  multivariate,  ANOVA.  There  was  a  significant  effect  of  connector 
type,  £  (3,  29)  =  14.65,  p  <  .001.  The  mean  proportions  and  the  transformed  values  are 
provided  in  Table  11.  Four  post  hoc  comparisons  (alpha  =  .0125)  indicated  that  additives  and 
causals  were  chosen  as  distractors  more  frequently  than  adversatives  and  sequentials,  £  (1 , 

31)  =  44.85,  p  <  .001,  Mserror  *  .772.  Causal  distractors  were  chosen  more  frequently  than 
additives,  £  (1,  31)  =  7.57,  £  =  .01,  Mserror  =  -605.  Adversative  and  sequential  distractors 
were  chosen  equally  often.  Confidence  ratings  on  the  incorrect  choices  did  not  significantly 
differ  across  the  four  connector  types,  £  (3,  60)  =  2.16,  p.  =  .1. 


Insert  Tables  11,  12  and  13  about  here 


The  results  of  analyzing  the  dependency  between  the  correct  connector  types  for  the  cloze 
slots  and  the  type  of  distractor  selected  when  errors  were  made  are  given  in  Table  12.  The 
pattern  replicates  the  pattern  observed  in  Experiment  1.  The  causal  connector  was  the 
dominant  distractor  for  additive  cloze  slots,  and  the  distributions  for  the  adversative  and 
sequential  cloze  slots  were  virtually  identical  to  those  given  in  Table  5.  The  one  difference 
between  the  results  of  the  two  experiments  is  that  more  adversative  distractors  were  chosen  in 
causal  slots  by  the  students  in  the  second  study. 


We  examined  the  selection  rate  data  for  evidence  of  a  choice  bias  toward  additive  and 
causal  distractors  and  the  selection  rate  data  are  shown  in  Table  13.  The  percentages  indicate  a 
bias  toward  causals  in  that  32%  of  choices  were  causals;  however,  only  66%  of  those  choices 
were  correct.  There  was  less  of  a  bias  toward  additives  than  in  Experiment  1 .  Adversatives  and 
sequentials  were  chosen  least  frequently  but  had  the  higher  percent  correct  rates.  Confidence 
ratings  for  correct  responses  were  highest  for  the  less  frequently  chosen  connector  types. 

Discussion 

The  results  of  Experiment  2  replicated  the  findings  of  Experiment  1  and  extended  them 
by  showing  that  students  were  more  confident  of  cloze  completions  for  adversative  and 
sequential  connectors.  Adversative  and  sequential  connectors  appear  to  have  more  restricted  and 
perhaps  clearer  usage  conditions  than  do  additives  and  causals.  Causals  and  additive  terms,  such 
as  and  and  so  may  be  more  frequent  in  everyday  language  and  their  usage  may  not  always 
literally  connote  a  highly  restricted  meaning  compared  to  adversatives  and  sequentials;  as  a 
result,  some  of  their  more  specific  meanings  may  become  "diluted"  in  text  more  so  than 
adversatives  and  sequential.  Students  may  choose  additives  and  causals  because  they  are  more 
familiar  in  everyday  speech,  not  because  they  are  certain  about  their  appropriateness  in  the 
contexts  of  written  text.  Hence,  even  when  their  choices  are  correct,  students’  confidence  in 
additive  and  causal  completions  is  weaker  than  their  confidence  in  less  frequently  chosen 
adversative  and  sequential  connectors. 

These  data  provide  support  for  the  interpretation  tnat  although  adversative  logical 
relations  and  sequential  temporal  relations  are  less  often  constructed  by  students,  when  they 
are  constructed  students  are  confident  of  the  accuracy  of  their  inferences.  There  is  significantly 
more  doubt  when  causals  and  additives  are  selected  for  use  in  a  text.  Experiment  3  pursued 
these  issues  with  ESL  students  at  generally  lower  levels  of  English  language  competency  than  the 
individuals  who  participated  in  Experiment  1.  Because  of  this,  we  also  used  a  new  sample  of 
texts  appropriate  to  this  population  and  a  somewhat  different  set  of  connector  terms. 
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EXPERIMENT  3:  Effects  of  Connector  Type  on  Cloze  Completions,  Confidence 
Ratings  and  Response  Justifications  of  ESL  Speakers 

The  purpose  of  Experiment  3  was  to  examine  whether  ESL  students'  confidence  in  their 
cloze  completions  varied  with  the  type  of  connector.  In  addition,  we  were  interested  in  testing 
the  replicability  of  the  ESL  results  from  Experiment  1 .  Due  to  constraints  on  the  ESL 
population  in  our  locale,  the  sample  for  Experiment  3  was  drawn  from  a  local  two-year, 
community  college  rather  than  from  the  university.  Because  this  population  is  generally  at 
lower  levels  of  English  language  proficiency  than  the  university  population,  we  developed  a  new 
set  of  passages  based  on  texts  at  the  level  of  the  students'  proficiency  in  English.  Thus, 

Experiment  3  was  a  replication/extension  of  the  basic  findings  of  Experiments  1  and  2  to  both  a 
new  subject  population  and  different  materials.  Data  were  collected  on  the  cloze  completion, 
the  confidence  rating  and  the  response  justification  tasks. 

Method 

Subsets 

Participants  were  35  ESL  students  enrolled  in  the  advanced-level  ESL  English  course  at 
a  local  community  college.  Nineteen  of  the  students  were  enrolled  in  a  course  that  met  during 
the  day  and  16  attended  a  night  class.  All  students  in  the  class  participated  in  the  paper  and 
pencil  part  of  the  task  and  it  was  conducted  during  class  time.  Following  the  group  session,  8 
students  from  the  day  class  and  each  of  the  night  class  students  were  individually  interviewed 
for  purposes  of  explaining  their  response  choices  on  the  first  two  passages  that  they  had  read. 

The  ESL  students  in  Experiment  3  were  more  heterogeneous  than  the  ESL  students  who 
participated  in  Experiment  1.  As  is  evident  from  the  data  in  Table  14,  the  subjects  in  the 
present  study  were  older  and  reflected  a  greater  age  range.  They  had  taken  fewer  related 
courses  and  the  level  of  English  language  skills  (M  *  5.3  grade  level  equivalent)  was  lower  than 
for  the  ESL  students  in  Experiment  1 ,  who  were  reading  at  levels  sufficient  to  enter  the 
university.  Experience  in  courses  related  to  the  passage  topics  was  similar  to  that  reported  by 
students  in  the  other  studies,  a  mean  of  2.34  but  only  15  students  had  taken  such  courses.  The 


other  20  did  not  report  any  courses  in  high  school  or  community  college  related  to  the  passage 
content.  Most  of  the  students  had  had  3  or  4  courses  at  the  community  college  prior  to  the  time 
of  participation  in  the  study. 
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Insert  Table  14  about  here 


Design  and  Materials 

The  design  was  a  mixed  factorial  in  which  class  (day  or  evening)  was  a  between-subjects 
factor  and  connector  type  (four  levels)  was  the  within-subjects  factor.  Dependent  measures 
were  derived  from  the  cloze  completion  responses,  the  confidence  rating  task  and  the 
justification  data.  Due  to  the  generally  lower  level  of  English  skill  in  this  population  of  ESL 
students  as  compared  to  the  university  students  of  Experiment  1 ,  new  passages  were  developed. 
The  same  constraints  on  the  occurrence  of  blanks  and  distribution  of  response  alternatives 
described  for  Experiment  1  and  2  were  followed  in  constructing  the  passages  used  in 
Experiment  3. 

Passages.  The  four  passages  were  modified  versions  of  texts  drawn  from  textbooks  that 
had  been  used  in  the  past  by  the  ESL  program  at  the  community  college  but  not  by  the  particular 
students  who  participated  in  the  study.  We  selected  passages  that  dealt  with  fields  in  the  social 
and  natural  sciences.  Specifically,  the  following  topics  were  discussed:  life  styles  during  the 
Paleolithic  Age,  emotions  that  cause  laughter,  the  natural  resources  in  Siberia,  and  the  m,stery 
of  the  Bermuda  triangle.  After  modification  to  accommodate  the  connectors,  the  mean  number  of 
words  per  passage  was  518,  with  the  shortest  507  words  in  length  and  the  longest  539;  mean 
number  of  sentences  per  passage  was  32.25,  with  the  shortest  having  28  sentences  and  the 
longest  38.  For  3  of  the  passages,  the  Flesch  grade  level  equivalent  was  9-10  and  for  1  it  was 
7-8. 

The  instances  of  the  four  types  of  connectors  used  in  the  new  passages  are  given  in  Table 
15.  As  with  the  four  passages  used  in  the  Experiments  1  and  2,  a  range  of  instances  were  used 
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for  each  connector  type  and  we  equalized  the  frequency  of  occurrence  o*  each  instance  as  much  as 
possible. 

Insert  Table  15  about  here 

A  passage  booklet  and  accompanying  response  booklet  were  created  for  each  subject.  The 
passages  were  presented  in  four  different  orders  to  counterbalance  position  effects  for  the 
experimental  passages.  The  practice  passage,  presented  first,  was  the  same  one  used  in 
Experiments  1  and  2.  As  in  Experiment  2,  each  page  of  the  response  booklet  contained  the  four 
alternative  choices  for  a  particular  cloze  slot  as  well  as  a  confidence  rating  scaie.  The  same 
constraints  described  for  Experiments  1  and  2  on  the  ordering  of  the  response  alternatives  were 
followed  in  this  experiment. 

Procedure 

The  subjects  enrolled  in  the  day  class  completed  the  passage  and  response  booklets  over  a 
series  of  4  class  sessions,  each  session  lasting  for  55  minutes.  The  subjects  enrolled  in  the 
evening  class  completed  their  booklets  over  a  series  of  2  class  sessions  each  approximately  2 
hours  in  duration.  The  procedure  involving  the  booklets  was  identical  to  the  one  used  in 
Experiment  2.  After  completing  the  booklets,  a  sample  of  8  students  from  the  day  class  were 
seen  in  individual  sessions  and  asked  to  explain  their  response  choices  for  the  first  two 
passages.  Subjects  were  systematically  selected  so  that  a  discussion  of  each  passage  occurred  an 
equal  number  of  times.  All  subjects  enrolled  in  the  evening  class  were  interviewed.  The 
interviewing  process  for  subjects  in  both  classes  began  once  all  subjects  in  the  class  had 
completed  the  booklets,  and  spanned  a  period  of  several  days  because  of  the  limited  time 
available  for  individual  sessions. 
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Results 

ConscLftes  ponses 

The  mean  probability  of  correctly  selecting  each  type  of  connector  was  computed  for  each 
student.6  A  split-plot,  multivariate  ANOVA  revealed  no  main  effect  (£  (1,  33)  =  2.93,  p.  > 

.05)  nor  interaction  {£  <  1)  involving  the  class  variable.  However,  consistent  with  • 

Experiments  1  and  2,  there  was  a  main  effect  of  connector  type,  £  (3,  31)  =  13.76,  p_<.  001. 

Five  post  hoc  comparisons  were  computed  using  the  means  shown  in  Table  16  and  the  Bonferroni 
procedure  was  used  to  control  the  Type  I  error  rate  (alpha  =  .01).  Additives  and  causals  were  • 

correctly  chosen  more  frequently  than  adversatives  and  sequentials,  £  (1,  33)  =  36.27,  p  < 

.001,  Mserror  =  -075.  Additives  and  causals  were  not  significantly  different,  £  (1,  33)  = 

3.47;  nor  were  adversatives  and  sequentials,  £  (1,  33)  =  3.45.  This  pattern  replicates  the  € 

results  obtained  in  Experiments  1  and  2;  Responses  for  additive  and  causal  slots  were  more 
often  correct  than  were  those  for  adversative  and  sequential  slots. 

Insert  Table  16  about  here 


In  contrast  to  the  findings  of  Experiment  2,  the  ANOVA  of  the  confidence  rating  data  failed 
to  reveal  any  significant  differences  due  to  connector  type,  £  (3,93)  =  2.19,  a  =  .09,  Mserror 
=  .24.  Thus,  the  ESL  students  were  no  more  confident  of  their  adversative  or  sequential  choices 
than  they  were  of  their  additive  and  causal  choices.  Furthermore,  their  confidence  ratings  for 
their  correct  answers  were  about  1  scale  value  below  the  ratings  of  the  native  English  speakers' 
confidence  in  their  correct  answers. 

Distractor.  Choices 

To  examine  whether  there  were  differences  among  the  connectors  in  the  likelihood  of 
being  selected  as  a  distractor,  we  computed  the  proportion  of  each  subject's  total  errors 
associated  with  each  distractor  alternatives.  These  proportions  were  then  transformed  using 
the  arcsine  transformation  and  submitted  to  a  multivariate,  split-plot  ANOVA  in  which  class  and 
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connector  type  were  factors.  As  in  the  analysis  of  the  correct  responses,  there  was  no  effect  of 
class,  £  (1,  33)  =  3.51,  a  =  .07.  There  was,  however,  a  significant  effect  of  connector  type,  £ 
(3,  31)  =  20.77,  a  <  .001.  The  means  are  shown  in  Table  17.  Five  post  hoc  comparisons, 
using  the  Bonferroni  procedure  (alpha  level  =  .01),  were  conducted.  Additive  and  causal 
distractors  were  selected  more  frequently  than  adversative  and  sequential  distractors,  £  (1, 

33)  =  51.77,  a  <  -001,  Mserror  =  -381.  Causal  distractors  were  selected  significantly  more 
often  lhan  additives,  £  (1,  33)  =  13.23,  a  <  .001,  Ms  error  =  -24.  The  probabilities  of 
choosing  adversative  and  sequential  distractors  were  not  significantly  different  from  one 
another;  nor  were  the  probabilities  of  choosing  additive  and  adversative  distractors.  The 
distractor  choice  data  replicate  the  findings  for  the  native  English  speakers  in  Experiments  1 
and  2  in  that  causals  were  the  most  frequently  chosen  distractor.  For  the  ESL  students  in 
Experiment  1  there  was  a  tendency  to  choose  the  additives  and  causals  most  often;  thus  there  was 
a  minor  difference  between  the  two  samples. 


Insert  Table  17  about  here 


ANOVA  on  the  confidence  ratings  for  the  incorrect  cloze  completions  revealed  a 
significant  effect  of  connector  type,  £  (3,  29)  =  3.96,  a  <  .01.  Post  hoc  contrasts  indicated  that 
confidence  was  higher  for  adversative  distractors  than  for  sequential  distractors,  £  (1,  31)  = 
11.24,  a  =  .002.  Examination  of  the  patterns  of  the  means  shown  in  Tables  16  and  17  suggested 
that  there  was  a  potentially  interesting  interaction  between  connector  type  and  ratings  for 
correct  compared  to  incorrect  choices.  The  ANOVA  on  these  data  indicated  higher  confidence 
when  the  correct  alternative  was  chosen  (M  =  4.93)  than  when  a  distractor  was  chosen  (M  = 
4.57),  £  (1,  28)  =  14.29,  a  *  01.  There  was  also  a  significant  connector  type  by  correctness 
interaction,  £  (3,  26)  =  3.64,  a  =  -02.  The  interaction  was  pursued  with  three  posthoc 
contrasts  (alpha  level  =  .017).  The  significant  contrasts  indicated  (1)  that  the  difference 
between  ratings  of  correct  and  incorrect  responses  was  larger  for  sequential  connectors  than  it 


was  for  the  other  three  connector  types,  £  (1 ,  28)  =  8.63,  ft  =  .007,  Mserror  =  8.57;  and  (2) 
that  correct  selections  of  sequentials  received  higher  confidence  ratings  that  when  sequential 
distractors  were  chosen,  £  (1,  28)  =  17.33,  ft  <  .001,  Mserror  =  .971. 

The  results  of  Experiments  1  and  2  suggested  that  both  native  English  and  ESL  students 
often  selected  causal  connectors  when  additives  were  most  appropriate.  In  the  present  sample 
of  ESL  students,  the  dependency  relation  analysis  indicated  a  much  weaker  relationship.  As  the 
data  in  Table  18  indicate,  50%  of  the  distractor  selections  for  additives  were  causals  but 
adversatives  were  selected  in  almost  40%  of  the  cases.  In  addition,  additive  and  adversative 
distractors  were  the  most  frequent  distractors  selected  when  errors  were  made  on  causal  slots; 
finally,  causals  were  the  most  frequently  selected  distractor  for  adversative  and  sequential 
slots.  It  appears  that  these  ESL  students  who  were  at  lower  levels  of  language  proficiency  had  a 
strong  tendency  to  overattribute  causality. 

Insert  Table  18  &  19  about  here 


That  there  was  a  general  bias  toward  selecting  causal  connectors  is  further  supported  by 
the  analysis  of  the  connectors  selected  by  the  students.  The  data  are  provided  in  Table  19. 

First,  34%  of  the  choices  were  causals,  reflecting  the  tendency  to  choose  causal  alternatives. 
However,  the  likelihood  that  these  choices  would  be  correct  was  relatively  low,  42%. 
Sequentials  were  least  often  chosen  but  had  the  highest  likelihood  of  being  correct.  Additives 
were  given  the  highest  confidence  ratings,  in  contrast  to  the  pattern  manifest  by  the  native 
English  speakers  in  Experiment  2.  Thus,  although  the  adversative  and  sequential  alternatives 
were  selected  least  often,  they  did  not  manifest  the  pattern  of  proportion  correct  and  confidence 
ratings  that  prevailed  in  Experiment  2.  Nor  did  the  present  sample  of  ESL  students  manifest  the 
pattern  shown  by  the  ESL  students  in  Experiment  1 ,  wherein  adversative  and  sequential 
alternatives  were  selected  least  frequently  but  were  most  frequently  correct.  It  is  possible  that 
these  ESL  students  who  are  at  less  sophisticated  levels  of  English  language  training  did  not  make 


use  of  the  greater  degree  of  constraint  governing  the  use  of  adversative  and  sequential 
connectors.  The  verbal  justifications  were  pursued  to  enlighten  the  criteria  that  these  students 
were,  in  fact,  using. 
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Justifications  of  Responses 

Justifications  for  Correct  Responses.  Table  20  provides  the  proportion  of  responses  in 
each  of  the  justification  categories  for  each  of  the  connectors.7  Several  trends  are  important. 
First,  the  majority  of  the  justifications  for  each  of  the  connectors  were  consistent  with 
definitions  and  taxonomies  of  logical  connectors,  .i.  e.,  60%  or  more  of  the  responses  were  in 
connector-appropriate  categories.  Furthermore,  the  dominant  responses  within  each  category 
were  consistent  with  the  dominant  responses  obtained  from  the  native  English  and  ESL  four- 
year  college  students  in  Experiment  1 .  Justifications  for  additive  slots  were  divided  between 
example  and  elaboration.  The  dominant  justification  for  causal  slots  was  the  existence  of  a 
cause-effect  relation;  the  dominant  justification  for  adversative  slots  indicated  that 
comparison,  contrast,  or  unexpected  information  was  present.  For  the  sequential  slots, 
justifications  involving  new  or  next  points  or  temporal  relations  were  given.  There  was  some 
difference  between  the  pattern  on  the  sequential  in  this  study  and  the  patterns  in  Experiment 
1 ;  however,  in  designing  the  texts  for  Experiment  3  we  purposely  tried  to  use  connectors  that 
indicated  temporal  relations  or  "next"  points.  Thus,  differences  in  the  sequential  justification 
data  between  Experiments  1  and  3  are  undoubtedly  due  to  the  specific  passages  and  connectors 
we  included  in  the  materials. 


Insert  Table  20  about  here 


There  was  one  difference  between  Experiments  1  and  3  that  was  probably  not  due  to  the 
specific  passages:  Approximately  37%  of  the  justifications  for  correct  responses  were  in  the 
Miscellaneous  category  and  the  majority  of  these  were  choice  by  exclusion,  i.e.,  "The  others 
didn't  fit."  The  higher  frequency  of  the  miscellaneous  category,  especially  making  a  choice  by 
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eliminating  the  other  options,  suggests  that  the  community  college  students  may  be  operating 
with  a  greater  degree  of  implicit  or  tacit  knowledge  about  these  connectors  than  are  the 
university  students. 


Justifications  for  the  incorrect  responses.  As  with  the  justifications  for  correct 
responses,  about  30%  of  the  justifications  of  incorrect  responses  were  in  the  miscellaneous 
category,  with  no  difference  between  connectors  in  this  trend.  Consistent  with  the  results  of 
Experiment  1 ,  the  proportion  of  justifications  for  choices  that  were  incorrect  generally 
reflected  the  dependency  relations  that  were  reflected  in  Table  16.  For  example,  36%  of  the 
44  justifications  for  incorrect  causal-slot  responses  were  additive- appropriate  reasons  and 
18%  were  adversative-appropriate.  For  adversative  slots,  27%  were  additive-appropriate 
and  27%  were  causal-appropriate  categories.  For  incorrectly  completed  sequential  slots, 
causal-appropriate  explanations  were  provided  41%  of  the  time.  The  additive  slots  were  the 
only  ones  were  subjects  showed  more  than  a  slight  tendency  to  use  connector-appropriate 
connectors:  23%  of  the  justifications  for  incorrect  responses  were  additive-appropriate. 

The  justification  data  were  used  in  conjunction  with  the  completion  data  to  examine  the 
source(s)  of  difficulty  for  these  students.  Of  the  three  relations  discussed  in  the  context  of 
Experiment  1  and  illustrated  in  Figure  1 ,  we  found  two  of  them  to  be  equally  likely  in  the  data  of 
the  ESI  community  college  students.  For  48%  of  the  justifications  for  incorrect  responses, 
students  used  a  justification  appropriate  to  the  distractor  they  had  selected  (branch  1  in  Figure 
1).  This  pattern  implies  difficulty  understanding  the  relation  called  for  by  the  text.  An 
additional  45%  of  the  justifications  for  incorrect  responses  used  a  justification  that  was 
inappropriate  given  the  slot  in  the  text  and  did  not  fit  the  distractor  selected  (branch  3  in 
Figure  1).  This  pattern  implies  difficulties  both  in  processing  the  relations  in  the  text  and  in 
understanding  the  meanings  of  specific  connector  terms  in  context.  Justifications  that  were 
inappropriate  to  the  choice  but  appropriate  to  the  slot  were  relatively  rare  (6%)  and  indicated 
that  knowledge  of  the  specific  connector  words  was  not  the  major  source  of  difficulty.  Thus,  the 
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major  problem  seemed  to  be  in  processing  logical  relations  in  context  and  recognizing  when  the 
specific  types  of  logical  connectors  were  needed. 

Discussion 

The  patterns  of  results  in  this  study  were  generally  consistent  with  our  predictions  and 
the  results  of  those  of  Experiments  1  and  2.  Cloze  completion  responses  were  more  frequently 
correct  for  additive  and  causal  slots  than  for  adversative  and  sequential  slots  and  justifications 
for  the  correct  responses  reflected  the  appropriate  connector  functions.  However,  contrary  to 
expectations,  the  ESL  students’  confidence  ratings  for  correct  choices  were  equivalent  to  one 
another.  When  they  were  incorrect,  ESL  students  most  frequently  chose  causal  distractors, 
although  their  confidence  in  these  choices  was  not  terribly  high.  The  confidence  rating  data  for 
the  incorrect  choices  was  unanticipated:  students  were  more  confident  of  wrong  answers  when 
they  had  chosen  adversatives  and  additives  than  when  they  had  chosen  causals  and  sequentials. 

Finally,  the  dependency  relations  between  the  correct  cloze  completion  and  the  type  of  connector 
incorrectly  selected,  in  conjunction  with  the  explanations  of  the  incorrect  choices,  suggest  that 
in  this  sample,  the  ESL  students  had  a  less  precise  understanding  of  the  differences  in  meaning 
between  frequently  used  causal  expressions,  such  as  thus,  so,  and  hence  and  often  confuse  them 
with  simpler  additive  or  sequential  indicators. 

GENERAL  DISCUSSION 

In  each  of  the  three  studies  there  was  a  consistent  effect  of  connector  type:  when  cloze 
slots  required  additive  or  causal  completion  terms,  students  were  more  likely  to  be  correct  than 
when  adversative  or  sequential  terms  were  needed.  We  had  predicted  that  sequentials  would  be 
difficult  because  (1)  there  are  a  greater  number  of  sequential  subtypes  and  (2)  correctly 
selecting  one  often  requires  a  reference  to  the  global  passage  rather  than  to  the  local  clause  or 
sentence  context.  We  were  somewhat  surprised  that  the  adversative  tended  to  be  as  difficult  as 
the  sequential.  We  suggest  three  plausible  explanations  of  performance  on  adversatives.  The 
specific  instances  of  the  adversative  may  have  lower  frequency  of  use  than  the  instances  of 
additives  and  causals.  Second,  performance  on  the  adversative  may  be  affected  by  the  existence 


of  a  reader  consistency  bias.  That  is,  readers  may  be  operating  with  a  default  assumption  that 
favors  interpreting  successive  sentences  as  elaborating  on  the  old  material  rather  than  by 
contradicting  it  or  restricting  its  scope.  Finally,  recognition  of  an  adversative  relation  may 
require  a  more  complex  backward  search  to  prior  content  than  the  causal  or  the  additive. 

The  effect  of  linguistic  proficiency  on  correct  performance  was  consistent  with  our 
general  expectations.  ESL  speakers  were  able  to  correctly  complete  fewer  cloze  slots  than 
native  English  speakers;  community  college  ESL  speakers  performed  at  lower  levels  than 
university  ESL  speakers.  However,  it  is  important  that  the  connector  type  pattern  was 
generally  consistent  across  groups. 

That  there  was  a  response  bias  toward  the  additive  and  causal  instances  was  illustrated  in 
two  of  the  measures.  First,  causal  and  additive  connectors  were  the  most  frequently  chosen 
distractors.  Furthermore,  the  student  selection  rate  data  showed  a  strong  tendency  to  choose 
causals  and  additives.  The  selection  rate  analysis  also  showed  that  adversative  and  sequentials 
were  more  likely  to  be  correct  when  they  were  selected,  and  students  had  the  highest  degrees  of 
confidence  in  these,  although  this  effect  was  not  as  strong  among  the  least  English  proficient. 

Analyses  of  the  incorrect  choices  revealed  some  interesting  information  regarding 
intersenteniial  reasoning  and  inference  making.  Causal  distractors  were  particularly  likely 
when  students  failed  to  choose  the  correct  additive  alternative.  Justifications  for  these  errors 
indicated  that  students  had  inferred  a  causal  relation  where  an  additive  had  been  intended.  The 
tendency  to  incorrectly  choose  a  causal  was  present  but  at  attenuated  levels  for  the  adversative 
and  sequential  slots.  The  strongest  trend  toward  choosing  the  causal  was  present  in  the  data  of 
the  community  college  ESL  students. 

The  patterns  associated  with  incorrect  responding  may  reflect  two  influences.  Readers 
may  be  using  the  causal  in  an  effort  to  create  relatively  tight  connections  among  units  of 
information.  Alternatively,  conversational  English  may  create  a  "sloppy"  meaning  for  causal 
connectors  such  as  so,  thus,  and  because.  That  is,  these  terms  may  be  used  in  situations  where  a 
relationship  other  than  cause-effect  is  being  discussed.  They  may  serve  as  "psuedo  bridges" 
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rather  than  as  true  causals.  As  a  result,  greatest  overuse  of  the  causal  would  be  expected  for 
those  students  whose  dominant  experiences  with  English  have  been  in  informal,  conversational 
contexts.  Similar  confusions  in  understanding  the  causal  have  been  reported  among  children 
acquiring  English  as  a  first  language  (e.g.,  Corrigan,  1975).  Content  domain  study  demands 
more  precision  in  the  meaning  of  the  language  we  use;  connectors  are  no  exception.  Thus,  as 
students  engage  in  more  interaction  with  formal  text,  they  are  forced  to  refine  their 
understandings  of  these  terms  and  the  usage  constraints  that  govern  them. 

The  relationship  between  incorrect  responses  and  their  justifications  indicated  that 
errors  were  predominantly  due  to  incorrect  inferences  about  the  appropriate  logical  relation 
and/or  an  inability  to  find  the  connector  that  expressed  that  relation  in  that  context.  When 
students  had  inferred  the  appropriate  relation  between  sentences,  they  rearely  erred  in 
choosing  the  correct  connector. 

Our  results  indicate  that  for  native  English  speakers  as  well  as  for  ESL  speakers  who 
have  attained  levels  of  English  proficiency  equivalent  to  about  the  fifth  grade,  there  are 
consistent  differences  among  connector  types.  The  correct  choice  justification  data  provide 
support  for  the  theoretically-based  taxonomic  distinctions  among  connector  types.  In 
processing  text,  readers  appear  to  assume  that  local  coherence  is  based  on  a  relatively 
generalized  relation  of  expanding  on  the  topic  introduced  by  the  previous  sentence.  The 
boundaries  between  a  clear  elaborative  relation  as  compared  to  a  bonafide  cause-effect  relation 
appear  to  be  somewhat  ill-defined  and  there  is  a  tendency  to  overattribute  the  cause-effect 
relation.  Logical  relations  that  indicate  contradiction  or  contrast  between  succesive  sentences 
in  text  are  more  difficult  for  students  to  identify.  When  they  do  select  an  adversative  connector 
there  is  a  very  high  probability  that  it  is  correct. 

Sequential  connectors  proved  to  be  the  most  difficult.  As  suggested  in  the  introduction 
the  reason  for  the  greater  difficulty  of  this  type  of  connector  may  be  that  it  requires  the  use  of 
more  than  local  discourse  context.  Many  of  the  sequential  connectors  used  in  the  present  studies 
require  that  the  student  keep  track  of  the  more  global  discourse  organization.  Furthermore, 
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sequential,  moreso  than  the  other  types  of  connectors,  are  often  used  to  signal  the  general 
organization  of  discourse  (cf.  Lorch,  in  press)  and  a  focus  only  on  the  local  context  does  not 
provide  sufficient  information  to  adequately  respond.  Sequentials  often  mark  the  sentence  they 
introduce  as  a  preview  or  review  of  text  information.  This  discourse  function  does  signify  a 
logical  relation  between  the  information  so  marked  and  the  remainder  of  the  text  but  it  does  not 
make  clear  the  specific  logical  relation  between  the  information  in  the  sentence  and  the 
information  in  the  just-prior  sentence.  When  students  answered  these  incorrectly  their 
distractor  choices  seemed  to  reflect  confusion  over  the  precise  nature  of  the  logical  relation: 
the  additive,  causal,  and  adversative  alternatives  were  all  chosen  with  some  frequency. 

A  major  source  of  difficulty  for  students  doing  this  task  was  inferring  the  appropriate 
relation  between  successive  sentences.  For  informational  texts  such  as  these,  designed  with  the 
explicit  purpose  of  communicating  new  information  to  students,  it  seems  particularly 
important  to  use  logical  connectors,  especially  when  contrastive  points  are  being  made.  It  is 
also  clear  that  in  content  domains  where  it  is  important  for  students  to  clearly  distinguish 
between  psuedo  and  true  cause-effect  relations,  explicit  connectors  in  the  text  will  facilitate 
accurate  understanding.  We  also  want  to  emphasize  that  differences  in  performance  between 
native  English  and  ESL  students  were  largely  in  overall  levels  of  performance  and  were  not 
primarily  associated  with  differential  patterns  among  the  connectors.  General  content-domain 
comprehension  skills,  rather  than  specific  connector  skills,  are  therefore  implicated  as  the 
locus  of  the  language  group  differences.  Once  ESL  students  have  grasped  the  basic  meaning  and 
functions  of  instances  of  specific  connectors,  further  drills  on  isolated  use  of  connectors  are  not 
likely  to  lead  to  improved  performance  on  connectors  in  natural  text  contexts.  Rather, 
improving  ESL  proficiency  at  this  level  seems  to  require  instruction  that  fosters  understanding 
logical  relationships  between  sentences  and  how  connectors  signal  such  relations  when  those 
sentences  occur  in  meaningful,  content-domain  contexts. 


33 


References 

Bridge,  C.  A.,  &  Winograd,  P.  N.  (1982).  Readers'  awareness  of  cohesive  relationships  during 
cloze  comprehension.  Journal  of  Reading  Behavior,  14,  299-312. 

Celce-Murcia,  M.,  &  Freeman,  D.  (1983).  The  grammar  book:  An  ESUEFL  teachers  course. 
Rowley,  MA:  Newberry  House. 

Corrigan,  R.  (1975).  A  Scalogram  analysis  of  the  development  of  the  use  and  comprehension  of 
"because"  in  children.  Child  Development,  46,  195-201. 

Geva,  E„  &  Ryan,  E.  B.  (1985).  Use  of  conjunctions  in  expository  texts  by  skilled  and  less 
skilled  readers.  Journal  of  Reading  Behavior,  17,  331-346. 

Goldman,  S.  R.,  &  Dur4n,  R.  P.  (1988).  Answering  questions  from  oceanography  texts: 

Learner,  task,  and  text  characteristics.  Discourse  Processes,  11,  373-412. 

Halliday,  M.  A.  K„  &  Hasan,  R.  (1976).  Cohesion  in  English.  London:  Longman. 

Lorch,  Jr.,  R.  F.  (in  press).  Text  signaling  devices  and  their  effects  on  reading  and  memory 
processes.  University  of  Kentucky,  Department  of  Psychology. 

MacLean,  M.  &.,  &  D’Anglejan,  A.  (1986).  Rational  cloze  and  retrospection:  Insights  into  first 
and  second  language  comprehension.  The  Canadian  Modern  Language  Review,  42,  (4, 

March).  Reprinted  in  G.  H.  Irons  (Ed.),  Selected  readings  in  theory  and  practice  (pp.  260- 
272).  Ontario,  Canada:  The  Canadian  Modem  Language  Review,  1988. 

Meyer,  B.  J.  F.  (1975).  The  organization  of  prose  and  its  effects  on  memory.  New  York: 
American  Elsevier  Pub. 

Meyer,  B.  J.  F.,  Brandt,  D.  M.,  &  Bluth,  G.  J.  (1980).  Use  of  top-level  structure  in  text:  Key 
for  reading  comprehension  of  ninth-grade  students.  Reading  Research  Quarterly,  16,  72- 
103. 

Spyridakis,  J.  H.,  &  Standal,  T.  C.  (1987).  Signals  in  expository  prose:  Effects  on  reading 


comprehension.  Reading  Research  Quarterly,  22,  285-298. 


34 


van  Dijk,  T.  E.,  &  Kintsch,  W.  (1983).  Strategies  of  discourse  comprehension.  New  York: 
Academic  Press. 

Voss,  J.  F.  (1984).  On  learning  and  learning  from  text.  In  H.  Mandl,  N.  L.  Stein,  &  T.  Trabasso 
(Ed.),  Learning  and  comprehension  of  text  (pp.  193-212).  Hillsdale,  NJ:  Erlbaum. 


35 


Appendix  A 

Categories  used  for  scoring  verbal  justification  data3 

Connector:  Additive 

1 .  Information  gives  example  of  concept. 

Second  sentence  states  an  example  or  particular  instance  of  a  concept  or 
issue  stated  in  the  first  sentence,  (e.g.,  "The  second  sentence  is  an  example  of 
how  it  interferes'*). 

2.  Information  elaborates  prior  information. 

The  second  sentence  elaborates  on  the  content  of  the  first  sentence  by 
stating  additional  related  information.  The  information  in  the  second  sentence 
might  be  described  as  providing  additional  consistent  information  to  the  point 
stated  in  the  first  sentence.  Also,  the  information  in  the  second  sentence  might 
be  described  as  being  supportive  of  or  strengthening  the  information  in  the  first 
sentence.  More  specifically,  subjects  might  say  things  such  as  "the  second 
sentence  explains  more  or  tells  more  about  the  first  sentence,  or  "This  sentence 
supports  the  first  sentence. 

Connector:  Adversative 

3b.  Comparison  or  contrast  with  information  in  the  prior  sentence;  unexpected 
information. 

The  second  sentence  states  a  comparison,  or  something  contrasting, 
opposite,  or  contradictory  relative  to  the  first  sentence.  Subjects  might  also 
refer  to  the  content  of  the  second  sentence  as  "something  unexpected".  This  is 
not  a  new  point  (such  as  in  Rule  7).  Instead,  the  content  of  the  second 
sentence  is  merely  different  relative  to  the  sentence  before  it. ...rather  than  part 
of  a  broader  superordinate  concept. 

Restriction  of  scope. 


3a. 


The  second  sentence  is  described  as  restricting  the  scope  of  the  previous 
(first)  sentence.  This  might  be  described  as  "limiting  the  boundaries  of  ..."  or 
"putting  a  restriction  on.."  the  content  of  the  first  sentence. 

Connector:  Causal 

4.  Cause  -  effect  relationship  between  the  two  sentences. 

The  second  senter  e  is  a  result,  effect  .or  consequence  of  something  that 
was  "set  up"  in  the  first  sentence.  The  important  thing  here  is  that  the  subject  is 
stating  that  there  is  a  cause  and  effect  relationship  going  on  between  the  first 
and  second  sentences.  How  the  subject  states  this  can  take  a  variety  of  forms: 

First  of  all,  the  subject  might  use  one  of  the  three  underlined  words  above;  in 
addition,  he/she  might  also  use  the  word  "because"  in  referring  to  the 
relationship  between  the  two  sentences  as  one  of  cause  and  effect  (e.g.,  "Because 
of  the  hard  water,  she  couldn't  wash  her  hair").  A  subject  might  also  refer  to  the 
content  of  the  second  sentence  as  a  conclusion  (in  a  different  way  from 
Justification  #6).  This  is  a  conclusion  about  cause  and  effect  (typically 
regarding  something  stated  only  1  or  2  sentences  back).  Be  careful  when 
subjects  use  the  word  "why".  They  might  use  "why"  to  refer  to  an  explanation 
about  the  previous  sentence  (in  that  case,  use  Justification  #11).  Or  they  might 
use  "why"  to  explain  a  cause  and  effect  situation  (and  if  so,  then  use  the  present 
category).  Whatever  words  the  subject  uses,  what  is  important  about  the 
Present  justification  is  that  subjects  state  that  their  choice  of  connecting  words 
was  because  of  a  cause  and  effect  situation. 

5.  Consistent  but  baguely  staed  logical  relationship. 

This  justification  should  be  used  when  the  subject  says  that  some  sort  of 
consistent,  logical  relationship  exists  between  the  first  and  second  sentences,  but 
is  not  explicit  (i.e.,  is  vague)  about  the  nature  of  that  relationship 
Usually  in  this  case,  subjects  say  something  like  "the  second  sentence  "follows" 
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from  the  first  sentence".  They  might  also  say  something  like  "the  first  sentence 
makes  you  expect  what  is  stated  in  the  second  sentence”;  or  "the  two  sentences  are 
just  “connected  together".  The  subject  might  also  use  *he  word  “why"  in  their 
justification.  In  such  a  case,  he\she  might  say  something  like  "The  second 
sentence  tells  why  the  first  sentence  was  stated"  or  something  equally  as  vague. 
The  critical  feature  for  this  category  is  that  the  subject  is  indicated  that  the  two 
sentences  are  logically  related,  although  s/he  is  not  stating  what  type  of 
relationship  exists.  If  the  subject  explicitly  mentions  a  sequential  or  causal 
relation  in  their  justification,  then  score  that  response  as  a  #9  or  #4, 
respectively. 

4&6.  Conclusion  about  a  cause-effect  relationship  that  was  developed  over  several 
sentences. 

The  second  sentence  states  a  conclusion  about  a  cause  and  effect 
relationship.  Uusally,  the  cause  and  effect  situation  is  expressed  over  several 
(prior)  sentences.  The  connector  introduces  the  final  sentence  in  the  cause  and 
effect  chain.  A  subject  might  say  something  like  "The  second  sentence  is 
concluding  why  a  (previous  event  stated  earlier  in  the  passage)  happened". 

Connector:  Sequential 

7.  Introduces  new  or  next  point. 

The  second  sentence  is  the  liisl,  second,  third,  final,  or  new  aspect  of  a 
multi-component  concept.  Subject  will  typically  make  reference  to  some  other 
part  of  the  passage  which  indicates  the  superordinate  concept  and/or  other 
components  that  are  also  listed.  Responses  can  be  classified  according  to  this 
justification  if  the  subject  says  that  the  word  signaled  a  new  pc.nt  (or  a 
numbered  or  final  point). 


9.  Temporal  relation 
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The  second  sentence  states  or  describes  a  second  event  that  occurs  after  a 
previous  event  in  the  first  sentence.  Typically,  a  subject  says  that  the 
relationship  between  the  two  sentences  is  one  of  "first  this  happens,  then 
this  other  thing  happens".  The  second  sentence  states  an  event  that  is 
temporally  distinct  from  an  event  stated  in  the  first  sentence.  If  the  subject 
states  that  a  causal  relationship  exists  between  the  2  events  use  Justification  #4 
If  they  mention  that  the  two  events  are  part  of  a  single  multicomponent  concept , 
use  justification  #7  If  they  indicate  the  existence  of  a  vague  logical  relationship 
between  the  events,  use  justification  5. 

6,8,  &11.  Sums  up  prior  or  previews  subsequent  information. 

There  were  several  categories  under  this  general  heading. 

6.  The  second  sentence  is  summing  up  previous  content,  or  making  a 
conclusion  about  previous  content.  This  content  might  be  an  entire  passage 
("This  is  concluding  or  summarizing  what  the  passage  was  about"),  or  it  might 
be  several  sentences  (This  is  summarizing  what  was  said  in  the  above  several 
sentences")  Be  careful  here  not  to  include  those  responses  that  are  making  a 
conclusion  that  pertains  to  a  cause  and  effect  relationship  involving  the  first  and 
second  sentences  -  use  justification  4  for  this  type  of  response.  This  kind  of 
conclusion  involves  a  larger  body  of  text,  i.e.,  it  is  a  more  general  conclusion. 
Subjects  might  also  say  that  this  is  a  good  word  choice  because  it  comes  near  the 
end  of  a  text.  Remember  also,  that  this  justification  doesn't  pertain  to 
introducing  a  final  or  concluding  point. 

8.  The  second  sentence  is  summing  up  what’s  to  come  later  on  in  the 
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passage.  This  kind  of  situation  can  also  exist  when  the  subject  lists  a  series  of 
components  or  issues  that  will  be  discussed  later  on  in  the  passage. 


1 1  .The  second  sentence  is  a  short  explanation  or  summary  of  what  was 
said  in  the  prior  (first)  sentence.  A  subject  might  state  that  the  second  sentence 
described  ,  defined,  or  summarized  what  just  was  said. 

10.  Miscellaneous 

If  a  response  cannot  be  categorized  as  adhering  to  one  of  the  above  itemized 
justifications,  then  it  is  classified  as  miscellaneous.  Included  in  this  category 
are  the  following: 

a.  Choice  by  exclusion:  "None  of  the  other  three  (alternatives)  worked"; 

b.  Guessing:  "I  just  guessed  at  this  one"; 

c.  Restating  or  paraphrasing  the  text 

d.  Metacognitive  or  affective  statements:  "It  was  easy  to  understand...", 
or  "It  made  me  confused." 


a  The  numbering  refers  to  the  original  categories  and  is  preserved  here  for  archival  purposes 
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Footnotes 

1 .  There  are  three  levels  of  ESL  classes  Linguistics  1 ,  Linguistics  2,  and  English  1 ,  however 
class  instructors  report  that  the  variability  within  the  class  is  often  greater  than  the 
variability  between  classes.  Following  successful  completion  of  English  1 ,  students  take  the 
English  composition  course  required  of  all  freshmen.  Four  students  came  from  Linguistics  2, 
and  one  from  Linguistics  1.  There  were  three  additional  students  (from  Linguistics  1)  who 
began  but  did  not  complete  the  task  because  of  their  inability  to  deal  with  the  level  of  English 
required  by  the  study. 

2.  No  ANOVA  was  done  to  examine  passage  effects  because  each  subject  provided  only  a  score  of 
0,  1 ,  or  2  for  each  connector  type  within  a  passage.  We  did  however  compute  the  number 
correct  on  each  connector  type  for  each  passage.  These  totals  indicated  relatively  equivalent 
performance  across  passages  for  the  additive,  adversative,  and  causal  connectors.  For  the 
sequential  connectors,  two  of  the  passages  produced  twice  as  many  correct  responses  as  in  the 
other  two  passages  for  both  native  English  and  ESL  speakers. 

3.  There  was  some  concern  that  the  verbal  justification  procedure  would  unfairly  discriminate 
against  the  ESL  speakers  in  terms  of  the  information  they  could  provide.  This  concern  seems 
unwarranted  because  only  3%  of  the  ESL  students’  responses  were  classified  in  the 
Miscellaneous  category;  3%  of  the  native  English  speakers'  responses  were  also  in  the 
Miscellaneous  category. 

4.  Students  were  not  told  whether  the  response  they  were  attempting  to  justify  was  correct  or 
incorrect.  Some  students  did  change  their  original  response  during  the  justification  phase  but 
the  overall  incidence  was  low:  4  for  the  native  English  speakers  (3  changed  from  an  incorrect 
to  a  correct  and  1  from  an  incorrect  to  a  different  incorrect  choice)  and  23  for  the  ESL.  About 
half  of  the  ESL  changes  were  from  an  incorrect  initial  choice  to  a  correct  final  choice;  30% 
went  from  correct  initial  to  incorrect  final  choices;  and  approximately  20%  were  changes  from 
one  incorrect  response  to  another.  For  purposes  of  analysis  of  the  number  correct  and 
distractor  choices  we  used  the  "final"  answer. 
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5.  Data  are  not  reported  separately  for  each  connector  because  the  trends  for  each  connector 
type  were  similar. 

6.  Probability  correct  was  analyzed  because  of  an  experimenter  error  in  constructing  the 
materials  -  there  were  only  7  usable  sequential  slots  for  the  day  class.  The  error  was  corrected 
in  the  materials  used  with  the  evening  class. 

7.  The  were  a  total  of  44  response  changes  during  the  justification  phase.  Half  of  these  were 
changes  from  an  incorrect  response  to  a  correct  response;  13%  were  from  a  correct  response  to 
an  incorrect  one;  and  37%  involved  changing  among  incorrect  responses. 


42 


Table  1 

Characteristics  of  the  Students  in  Experiment  1 a 


Characteristic 

Native  English 
Speakers 

ESL 

Speakers 

/*ge 

Mean 

18.75 

20.3 

Range 

18-20 

18-30 

Age  at  which  schooling 

Mean 

-  - 

1 1 .63 

in  English  began 

Range 

-  - 

4  -  26 

Scholastic  Aptitude  Test  (V)b 

Mean 

526.8 

338.3 

Range 

450  -  600 

210  - 

Social  Science  Courses 

Mean 

2.78 

3.1 

in  High  School  or  College 

Range 

0  -  12 

0  -  15 

Major  Field  of  Study 

Social  Science 

31% 

30% 

Natural  Science,  including  Engineering 

1  8% 

60% 

Undecided 

50% 

5% 

Native  Language 

English 

100% 

0% 

Spanish 

5% 

European  (except  Spanish) 

25% 

Asian  7  0  % 


(table  continues) 
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Table  1  icont'd) 

a  All  data  are  self-reported.  We  did  not  have  access  to  official  student  records. 
b  All  16  Native  English  speakers  reported  Verbal  SAT  scores;  only  12  ESL  speakers  reported 
Verbal  SAT  scores.  The  mean  SAT  score  for  freshmen  at  this  university  has  fluctuated  around 
500  for  the  past  several  years.  An  additional  7  ESL  students  reported  scores  on  the  Test  of 
English  as  a  Foreign  Language  (TOEFL)  and  2  students  reported  both  SAT  and  TOEFL  scores.  The 
mean  TOEFL  was  564.7,  with  scores  ranging  from  510  to  630.  The  mean  TOEFL  score  is 
representative  of  that  reflected  in  the  nonnative  English  speaking  undergraduate  population  at 
this  university. 
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Table  2 

Instances  of  Connectors  and  Frequency  of  Use  as  Correct  Responses  for  the  Passages  used  in 
Experiments  1  and  2 


Additive 

Adversative 

Causal 

Sequential 

in  addition 

2 

nevertheless 

2 

thus 

2 

briefly 

1 

for  example 

1 

but 

3 

consequently 

1 

first 

2 

for  instance 

1 

however 

3 

as  a  result 

1 

finally 

2 

in  particular 

1 

as  a  consequence 

1 

in  short 

1 

moreover 

1 

so 

1 

second 

1 

in  fact 

1 

therefore 

2 

third 

1 

indeed 

1 
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Table  3 

Mean  Number  Correct  on  each  Type  of  Connector  in  Experiment  1a 

Language  Group 


Native  English 

ESL 

Overall  Connector 

Connector 

(n-  16) 

(n  =  20) 

Means  (n  =  36) 

Additive 

6.44 

5.7 

6.03 

Causal 

6.56 

5.0 

5.69 

Adversative 

6.19 

4.35 

5.17 

Sequential 

6.19 

4.45 

5.22 

Overall  Group  Means 

6.34 

4.88 

aThe  overall  means  have  been  weighted  to  reflect  the  unequal  sample  sizes.  The  maximum 
number  correct  was  8. 


Table  4 


Distractor  selections  in  Experiment  1a 


Language  Group 


Native  English 

ESL 

(n*  16) 

(n  =  20) 

Mean  errorsb 

6.62 

12.20 

Distractor  Selected 

Additive 

.29 

.39 

Causal 

.41 

.31 

Adversative 

.05 

.14 

Sequential 

.16 

.13 

aThe  Anova  was  done  on  arcsine  transforms  of  the  proportion  of  incorrect  responses  for  which 
each  type  of  connector  was  selected.  The  means  of  the  transformed  measures  were  as  follows. 
For  the  native  English  speakers:  additive  =  1.1;  causal  =  1.38;  adversative  =  .44;  sequential  = 
.76.  For  the  ESL  speakers:  additive  =  1.37;  adversative  =  .73;  causal  =  1.16;  sequential  =  .7 
bThe  maximum  number  of  errors  was  32  (8  for  each  of  4  connectors). 
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Table  5 

The  Proportion  of  Errors  on  each  Connector  Type  Accounted  for  by  each  of  the  Distractor  Options 
for  Experiment  1.a 

Distractor  Options 


Correct  Connector 

Additive 

Causal 

Adversative 

Sequel 

Native  English  Speakers 

Additive 

.  .  . 

.76 

.08 

.16 

Causal 

.43 

-  -  - 

.09 

.47 

Adversative 

.40 

.43 

-  -  - 

.17 

Sequential 

.38 

.41 

.21 

-  - 

ESL  Speakers 

Additive 

-  -  - 

.60 

.24 

.15 

Causal 

.47 

— 

.15 

.38 

Adversative 

.49 

.40 

-  -  - 

.11 

Sequential 

.50 

.29 

.21 

-  - 

aThe  proportions  are  based  on  the  total  number  of  errors  for  each  connector  type.  For  the 
Native  English  speakers  the  number  of  errors  were  the  following:  On  additive  slots,  25;  on 
causal,  23;  on  adversative,  30;  on  the  sequential  slots,  29.  For  the  ESL  speakers  the  number  of 
errors  were  as  follows:  additive  ,  46;  causal  ,  60;  adversative  ,  72;  and  sequential  ,  70. 


Table  6 


Student  Selection  Rate  Distributions  for  Experiment  1 


Connector 


Percentage  of  Selections3 
Native  English  ESL 


Percentage  of  Selections 
Scored  as  Correct 
Native  English  ES 


Additives 

27 

33 

76 

Causals 

29 

28 

70 

Adversative 

21 

1  9 

91 

Sequential 

23 

20 

83 

aTotal  number  of  selections  for  the  Native  Enalish  students  was  512  and  the  total  for  the  ESL 


students  was  640. 
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Table  7 

Proportions  of  Justifications  in  each  Category  for  Correct  Responses  in  Experiment  1a 

Language  Group 


Connector  and  Justification  Category3  Native  English  ESL 

Additives  -  Total  number  of  correct  responses  4  6  5  5 

Information  gives  example  of  concept  .54  .40 

Information  elaborates  prior  information  .24  .42 

Inappropriate  justification  .17  .15 

Causals  -  Total  number  of  correct  responses  51  4  6 

Cause  -  effect  relationship  between  the  two  sentences  .71  .67 

Consistent  but  vaguely  stated  logical  relationship  .08  .04 

Conclusion  about  a  cause  -  effect  relationship  that  was 
developed  over  several  sentences.  .06  0 

Inappropriate  justification  .17  .22 

Adversatives  -  Total  number  of  correct  responses  4  5  4  6 

Comparison  or  contrast  with  information  in  the 
prior  sentence;  unexpected  information  .84  .91 

Restriction  of  the  scope  of  the  prior  sentence  .09  .04 

Inappropriate  justification  .04  .02 

Sequentials  -  Total  number  of  correct  responses  4  4  4  4 

Introduces  new  or  next  point  .77  .75 

Temporal  relation  0  0 

Sums  up  prior  or  previews  subsequent  information  .20  .23 

Inappropriate  justification  0  0 

Miscellaneous  (exclusion,  guessing,  restating  text)  .03  .03 


aSee  Appendix  A  for  a  full  description  of  the  justification  categories. 


Table  8 

Proportions  of  Justifications  in  each  Category  for  Incorrect  Responses  in  Experiment  1 

Type  of  Incorrect  Response  Justification  Appropriate  to 

Being  Justified  Frequency  Additive  Causal  Adversative  Sequential  Misc. 


Additive 
Native  English 
ESL 
Causal 


Native  English 

9 

.44 

.22 

ESL 

32 

.44 

.22 

Adversative 

Native  English 

1  6 

.13 

.56 

ESL 

32 

.34 

.34 

Sequential 

Native  English 

1  7 

.24 

.35 

ESL 

35 

.34 

.23 
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Table  9 


Characteristics  of  the  Students  in  Experiment  2a 

Characteristic 

Mean 

Range 

Age 

18.9 

17  -  24 

Scholastic  Aptitude  Test  (Verbal)b 

568.27 

440  -  700 

Social  Science  Courses  in  High  School  or  College*3 

3.03 

0  -  13 

Major  Field  of  Study 

Social  Science 

4  3% 

Natural  Science,  including  Engineering 

2  2% 

Undecided 

28% 

a  All  data  are  self-reported.  We  did  not  have  access  to  official  student  records. 
b29  Native  English  speakers  reported  verbal  Scholastic  Aptitude  Test  scores. 


i 


> 
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Table  10 

Mean  Number  Correct  on  each  Type  of  Connector  and  Mean  Confidence  Ratings  for  Correct  items 
in  Experiment  2a 


Connector 

Mean 

Mean  Confidence 

Correct 

Rating  for  Correct 

Additive 

5.88 

5.54 

Causal 

6.66 

5.67 

Adversative 

5,63 

5.98 

Sequential 

5.22 

5.97 

aThe  means  are  based  on  32  subjects  and  the  maximum  score  was  8  for  correct 
responses.  Confidence  ratings  are  on  a  7-point  scale  with  1  =  very  low  confidence  and  7 
=  very  high  confidence. 
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Table  11 

Distractor  selections  and  mean  confidence  ratings  in  Experiment  2 


Distractor 

Proportion  of 

Mean  Confidence 

Incorrect  Responses3 

Rating  for  lncorrectb 

Additive 

.27 

5.04 

Causal 

.42 

5.39 

Adversative 

.13 

5.07 

Sequential 

.1  6 

4.97 

aThe  mean  errors  per  subject  was  8.62  out  of  a  maximum  of  32.  The  Anova  was  conducted  on 
the  arcsine  transformations  of  the  proportion  of  incorrect  responses.  The  means  of  the 
transformed  proportions  were  as  follows:  additive  =  1.06;  causal  =1.44;  adversative  =  .68; 
sequential  =  .77. 

bThe  means  for  the  confidence  ratings  were  based  on  a  7-point  scale  with  1  =  very  low 
confidence  and  7  =  very  high  confidence.  The  means  are  based  on  21  subjects  because  1 1  did  not 
have  data  in  all  4  cells. 
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Table  12 

The  Proportion  of  Errors  on  each  Connector  Type  Accounted  for  by  each  of  the  Distractor 
Options  in  Experiment  2a 


Distractor  Options 


Correct  Connector 

Additive 

Causal 

Adversative 

Sequential 

Additive 

— 

.60 

.13 

.26 

Causal 

.27 

.  .  . 

.27 

.44 

Adversative 

.46 

.42 

.  .  . 

.1 1 

Sequential 

.37 

.42 

.20 

... 

aThe  proportions  are  based  on  the  total  number  of  errors  for  each  connector  type.  The  number 
of  errors  were  the  following:  On  additive  slots,  68;  on  causal,  43;  on  adversative,  76;  and  on 
the  sequential,  89. 
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Table  13 

Student  Selection  Rate  Distributions  for  Experiment  2 


Connector 

Percentage 

of  Selections3 

Percentage  of  Selections 

Scored  as  Correct 

Mean  Confidence 

for  Correct  Responses 

Additives 

26 

70 

5.54 

Causals 

32 

66 

5.64 

Adversative 

21 

82 

5.96 

Sequential 

21 

79 

5.97 

aTotal  number  of  selections  was  1024. 
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Table  14 

Characteristics  of  the  Students  in  Experiment  3a 


Characteristic 

Mean 

Range 

Age 

27.8 

19  -  65 

Age  at  which  school  ing  in  English  began 

20.6 

3  -  60 

Stanford  Diagnostic  Reading  Test,  Grade  Equivalent 

5.30 

2.7  -  10.6 

Social  Science  Courses  in  High  School  or  Collegeb 

2.34 

0  -  28 

Native  Language 

Spanish 

48.5% 

European  (except  Spanish) 

8.5% 

Asian 

43.0% 

aData  are  self-reported  except  for  the  Stanford  Diagnostic  Reading  Test  (SDRT)  Brown,  Form  A. 
The  SDRT  is  administered  by  the  community  college  for  placement  purposes  when  students 
enroll.  Scores  were  available  on  all  students.  Only  2  students  reported  a  TOEFL  score 
bOnly  15  of  the  35  students  had  taken  courses  related  to  Social  Sciences  in  High  School  and  only 
9  had  taken  such  courses  in  community  college. 
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Table  15 

Instances  of  Connectors  and  Frequency  of  Usage  as  Correct  Responses  for  the  Passages  used  in 
Experiment  3 


Additive  Adversative  Causal 


in  fact 

1 

in  contrast 

1 

thus 

1 

for  example 

1 

however 

1 

as  a  result 

2 

for  instance 

1 

on  the  other  hand 

1 

hence 

1 

also 

1 

in  comparison 

1 

as  a  consequence 

1 

moreover 

1 

nevertheless 

1 

consequently 

1 

as  an  example 

1 

yet 

1 

therefore 

1 

that  is 

1 

nonetheless 

1 

for  this  reason 

1 

Sequential 
first  1 

next  1 

briefly  2 

third  1 

later  1 

second  1 

in  short  1 


furthermore 


1 


rather 


1 


Table  16 


Mean  Probability  of  a  Correct  Response,  Mean  Number  Correct,  and  Mean  Confidence  Ratings  for 
Correct  items  in  Experiment  3a 


Mean 

Mean  Number 

Mean  Confidence 

Connector 

Probability 

Correct 

Ratings  for  Correct13 

Additive 

.61 

4.87 

5.1 1 

Causal 

.53 

4.72 

4.92 

Adversative 

.46 

3.76 

4.99 

Sequential 

.39 

2.89 

5.21 

aThe  maximum  number  correct  was  8  per  connector  type,  except  for  the  sequential 
connector.  The  maximum  correct  for  the  sequential  connector  was  7.46  because  for  1 9 
of  the  students  one  of  the  sequential  slots  was  discounted  due  to  a  typographical  error  in 
the  passage.  The  error  was  corrected  for  the  other  1 6  students.  The  data  for  probability 
and  mean  number  correct  are  weighted  appropriately.  Due  to  this  experimenter-error, 
the  ANOVA  was  done  on  the  probability  correct  scores. 

bThe  means  are  based  on  the  32  students  who  had  at  least  one  correct  response  for  each  of 
the  connectors.  Confidence  ratings  are  based  on  a  7-point  scale  with  1  =»  very  low 
confidence  and  7  -  very  high  confidence. 
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Table  17 

Distractor  Selections  and  Mean  Confidence  Ratings  in  Experiment  3 


Distractor 

Proportion  of 

Incorrect  Responses3 

Mean  Confidence 

Rating  for  Incorrect15 

Additive 

.25 

4.76 

Causal 

.39 

4.63 

Adversative 

.20 

4.82 

Sequential 

.13 

4.50 

aThe  mean  errors  per  subject  was  15.77  out  of  a  maximum  of  32  The  Anova  was 
conducted  on  the  arcsine  transformations  of  the  proportion  of  incorrect  responses.  The 
means  of  the  transformed  proportions  were  as  follows:  additive  =  1.05;  causal  =  1.36; 
adversative  =  .91;  sequential  =  .72. 

bThree  subjects  did  not  have  data  in  all  4  cells  and  the  means  are  based  on  32  subjects. 
Confidence  ratings  were  based  on  a  7-point  scale  with  1  =  very  low  confidence  and  7  =  very 
high  confidence. 


» 
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Table  18 


The  Proportion  of  Errors  on  each  Connector  Type  Accounted  for  by  each  of  the  Distractor  Options 
for  Experiment  3a 


Correct  Connector 

Additive 

Distractor  Options 

Causal  Adversative 

Sequential 

Additive 

— 

.50 

.39 

.10 

Causal 

.45 

-  -  - 

.37 

.17 

Adversative 

.28 

.44 

.  .  . 

.26 

Sequential 

.27 

.53 

.19 

... 

aThe  proportions  are  based  on  the  total  number  of  errors  for  each  connector  type.  The  number 
of  errors  were  the  following:  On  additive  slots,  108;  on  causal,  132;  on  adversative,  152;  and 
on  the  sequentials,  170.  The  number  of  possible  errors  was  280  for  all  but  the  sequential 
slots,  which  had  a  maximum  of  261. 


Table  19 


Student  Selection  Rate  Distributions  for  Experiment  3 


Connector 

Percentage 

of  Selections3 

Percentage  of  Selections 

Scored  as  Correct 

Mean  Confidence  for 

Correct  Responses 

Additives 

29 

53 

5.27 

Causals 

34 

42 

5.02 

Adversative 

23 

51 

5.09 

Sequential 

1  6 

57 

5.14 

aTotal  number  of  selections  was  1098,  representing  32  blanks  for  16  subjects  and  31  for  19 
subjects,  less  three  instances  of  no  response. 
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Table  20 

Proportions  of  Justifications  in  each  Category  for  Correct  Responses  in  Experiment  3 


Connector  and  Justification  Category3 
Additives  -  Total  number  of  correct  responses 
Information  gives  example  of  concept 
Information  elaborates  prior  information 
Inappropriate  justification 
Causals  -  Total  number  of  correct  responses 

Cause  -  effect  relationship  between  the  two  sentences 
Consistent  but  vaguely  stated  logical  relationship 
Conclusion  about  a  cause  -  effect  relationship  that  was 
developed  over  several  sentences. 

Inappropriate  justification 
Adversatives  -  Total  number  of  correct  responses 
Comparison  or  contrast  with  information  in  the 
prior  sentence;  unexpected  information 
Restriction  of  the  scope  of  the  prior  sentence 
Inappropriate  justification 
Sequentials  -  Total  number  of  correct  responses 
Introduces  new  or  next  point 
Temporal  relation 

Sums  up  prior  or  previews  subsequent  information 
Inappropriate  justification 
Miscellaneous*3  (exclusion,  guessing,  restating  text) 


aSee  appendix  for  full  description  of  the  justification  categories. 


(table  continues) 


Table  20  (cont'd.) 

Miscellaneous  did  not  differ  by  connector  type  and  the  .37  represents  the  rate  of  this 
response  over  all  connector  types. 


For  Incorrect  Cloze  Completion  Choices: 


Was  the  Justification  appropriate 
to  the  choice? 


understanding 


Was  the  Justification  appropriate 
to  the  cloze  slot? 


Yes 


No 


Faulty  connector  Faulty  text  and 

understanding  faulty  connector 

understanding 


Figure  1:  Relationships  between  incorrect  cloze  completion 
and  verbal  justifications 
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